***CONTrOL*** *is an agent-based model developed for the purpose of studying behavioral mechanisms influencing social learning and performance within and across business organisations. CONTrOL stands for Complex Organisational and Network-driven Transmissions resulting in Organisation Learning.*

# Landscape package

## Submodules

## landscape.generate_landscape module

This python code package serves to create an NK fitness landscape graph with an limited number of edges connecting the graph nodes (i.e. landscape locations).

### NK matrix generation

The `imatrix_rand(N,K)` method is a NumPy method that generates a random interaction matrix with N entities (=decision variables) and where each entity interacts with a specified number of other entities (K interactions). `N` is the number of rows and columns in the interaction matrix, which represents entities that interact with each other. `K` is a parameter that controls the number of interactions each entity has with other entities.
* It creates an empty square matrix `Int_matrix_rand` of size NxN, where each element is initialized to 0. This matrix will store the binary interactions between entities.
* It iterates through each entity (indexed by `aa1`) from 0 to N-1.
* For each entity (`aa1`), it creates a list `Indexes_1` containing all entity indices from 0 to N-1.
* It removes the current entity (`aa1`) from `Indexes_1` to avoid self-interactions.
* It shuffles the order of indices in `Indexes_1` randomly using `np.random.shuffle(Indexes_1)`.
* It appends the current entity (`aa1`) to the end of the shuffled list, ensuring that the entity interacts with itself.
* It selects the last (K+1) indices from the shuffled list (excluding the current entity `aa1`) and stores them in the `Chosen_ones` list. This determines which entities the current entity will interact with.
* For each selected entity (`aa2`) in `Chosen_ones`, it sets the corresponding element in the `Int_matrix_rand` to 1. This indicates that the current entity (`aa1`) interacts with the selected entities.
* The function returns the resulting binary interaction matrix `Int_matrix_rand`.
```python
def imatrix_rand(N, K):
    Int_matrix_rand = np.zeros((N, N))
    for aa1 in np.arange(N):
        Indexes_1 = list(range(N))
        Indexes_1.remove(aa1)
        np.random.shuffle(Indexes_1)
        Indexes_1.append(aa1)
        Chosen_ones = Indexes_1[-(K+1):]
        for aa2 in Chosen_ones:
            Int_matrix_rand[aa1, aa2] = 1 
    return(Int_matrix_rand)
```

### Fitness value calculation

The `calc_fit` function computes the fitness values for a set of decision variables based on a given combination and a fitness landscape (represented by `NK_land_`) with interactions and specific indexing. The function returns a vector of fitness values, with one value for each decision variable.
* It creates an empty array `Fit_vector` of size `N`, which will store the fitness values for each decision variable.
* It iterates through each decision variable (indexed by `ad1`) from 0 to N-1.
* For each decision variable, it calculates a fitness value using the following steps:
   * It constructs an index by performing element-wise multiplication between `Current_position`, `inter_m[ad1]`, and `Power_key_`. This index identifies a specific location or value in the `NK_land_` landscape.
   * It uses the calculated index to access the corresponding fitness value from the `NK_land_ landscape` and assigns it to the corresponding element in the `Fit_vector`.
* After calculating the fitness values for all decision variables, it returns the `Fit_vector`, which is a vector of fitness values corresponding to the provided combination of decision variables.
```python
def calc_fit(N, K, NK_land_, inter_m, Current_position, Power_key_):
    Fit_vector = np.zeros(N)
    for ad1 in np.arange(N):
        Fit_vector[ad1] = NK_land_[np.sum(Current_position * inter_m[ad1]
                                          * Power_key_), ad1]
    return(Fit_vector)
```

The `comb_and_values(N, K, NK_land_, Power_key_, inter_m)` function calculates values for all combinations on the landscape. It returns an array where each row represents a combination of decision variables, and the columns have the following meanings:
* The first `N` columns (indexed from 0 to N-1) represent each of the combinations of decision variables.
* The column indexed `N` is for the total fitness (the average of the entire fitness vector for that combination).
* The column indexed `N+1` represents the ID of the location.
* The column indexed `N+2` is a dummy variable, with 1 indicating a local peak.
* The last column is a dummy variable, with 1 indicating the global peak.

The function systematically explores all combinations of decision variables on a landscape, calculates their fitness, and determines whether each combination is a local peak or the global peak. 
* It creates an empty array `Comb_and_value` to capture the results. The size of this array is (2^N, N+4) to accommodate all possible combinations and associated values.
* It initializes two counters, `c1` and `c2`, which will be used to iterate through all combinations.
* It uses `itertools.product` to iterate over all possible combinations of binary values (0 and 1) for the N decision variables. For each combination, it performs the following steps:
   * Calculates the fitness of the combination using the `calc_fit` function.
   * Stores the combination and fitness values in the `Comb_and_value` array.
   * Calculates and stores the average fitness value in the N column of the array.
   * Assigns a unique ID (location reference) to the combination in the N+1 column.
* It checks if each combination is a local peak by comparing its fitness with the fitness of its neighbors. For each combination (c3), it iterates through each neighbouring decision variable (c4) and checks if flipping the value of that variable results in a fitness value greater than the fitness of the current combination. If any of the neighbors has higher fitness, the combination is not a local peak, and the N+2 column is set to 0 (indicating it's not a local peak).
* It identifies a global peak by finding the combination with the highest average fitness in the N column and sets the N+3 column to 1 for that combination.
* Finally, it returns the Comb_and_value array with all the calculated values.
```python
def comb_and_values(N, K, NK_land_, Power_key_, inter_m):
    Comb_and_value = np.zeros((2**N, N+4))  
    c1 = 0  
    for c2 in itertools.product(range(2), repeat=N):
        Combination1 = np.array(c2)  
        fit_1 = calc_fit(N, K, NK_land_, inter_m, Combination1, Power_key_)
        Comb_and_value[c1, :N] = Combination1  
        Comb_and_value[c1, N] = np.mean(fit_1)
        Comb_and_value[c1, N+1] = c1
        c1 = c1 + 1
    for c3 in np.arange(2**N): 
        loc_p = 1 
        for c4 in np.arange(N): 
            new_comb = Comb_and_value[c3, :N].copy().astype(int)
            new_comb[c4] = abs(new_comb[c4] - 1)
            if ((Comb_and_value[c3, N] <
                 Comb_and_value[np.sum(new_comb*Power_key_), N])):
                loc_p = 0 
        Comb_and_value[c3, N+2] = loc_p
    max_ind = np.argmax(Comb_and_value[:, N])
    Comb_and_value[max_ind, N+3] = 1
    return(Comb_and_value)
```

### NK landscape generation

The `generate_nk_landscape(N, K, name, edge_removal_percentage, Power_key_)` function generates an NK landscape and saves it as a NumPy array.
* It generates a random interaction matrix `Int_matrix` using the `imatrix_rand(N, K)` function.
* It creates a random NK landscape `NK_land` as a NumPy array of shape (2^N, N) with random values between 0 and 1.
* It calculates the landscape data by calling the `comb_and_values(N, K, NK_land, Power_key_, Int_matrix)` function, which calculates values for all combinations on the landscape, identifies local peaks, and the global peak. The resulting data is stored in a `Landscape_data` object.
* It constructs a filename for the landscape data using the provided name, `N`, `K`, and `edge_removal_percentage` (which is a parameter set **WHERE**), and then saves the `Landscape_data` as a NumPy binary file (.npy) to the specified directory.
* Finally, it returns the `Landscape_data`, which contains information about the landscape, combinations, and peak identifications.
```python
def generate_nk_landscape(N, K, name, edge_removal_percentage, Power_key_):
    Int_matrix = imatrix_rand(N, K).astype(int)
    NK_land = np.random.rand(2**N, N)  
    
    Landscape_data = comb_and_values(N, K, NK_land, Power_key_, Int_matrix)

    extra_naming = name+'_' if name != "" else ""
    np.save('../data/landscape/'+extra_naming+'N'+str(N)+'K'+str(K)+'_'+str(int(edge_removal_percentage*100))+'_raw.npy', Landscape_data)

    return Landscape_data
```

The `generate_landscape(N, K, edge_removal_percentage, name)` function generates an NK landscape, constructs a network based on the landscape, removes a specified percentage of edges, and saves the network data in a GEXF file. It takes the following arguments:
* `N` is the number of decision variables (entities).
* `K` is a parameter that controls the number of interactions each entity has with other entities.
* `edge_removal_percentage` is a parameter indicating the percentage of edges to remove from the network (as a fraction, e.g., 0.1 for 10%).
* `name` is a string used to name the landscape.

The function operates as follows:
* It calculates a `Power_key array`, which is used in the landscape generation. This array contains powers of 2, starting from 2^(N-1) down to 2^0, and it is used for indexing purposes.
* It generates an NK landscape using the `generate_nk_landscape(N, K, name, edge_removal_percentage, Power_key)` function. This generates the landscape data and saves it to a .npy file.
* It initializes a graph object `G` using NetworkX (`nx.Graph()`), which will represent the network of landscape combinations.
* It constructs an edge list `edgelist` that will store information about the edges in the network.
* It iterates over all combinations (represented by integers from 0 to 2^N) and their neighboring combinations to build the `edgelist`. The purpose of this loop is to create edges between combinations based on their similarity.
* It defines node attributes using the fitness values from the landscape data and associates these attributes with nodes in the graph to create the `node_attrs` dictionary.
* It adds edges to the graph `G` based on the `edgelist`.
* It calculates the number of edges to remove from the network based on the specified `edge_removal_percentage` and randomly selects edges to remove.
* It writes the resulting network `G` to a .gexf file, which is a file format used to represent graph data.
```python
def generate_landscape(N, K, edge_removal_percentage, name):
    Power_key = np.power(2, np.arange(N - 1, -1, -1))

    nk_data = generate_nk_landscape(N, K, name, edge_removal_percentage, Power_key)
    G = nx.Graph()
    edgelist = []

    for comb in np.arange(2**N):
        for neighbour in np.arange(N):
            new_comb = nk_data[comb, :N].copy().astype(int)
            new_comb[neighbour] = abs(new_comb[neighbour] - 1)
            if (int(nk_data[np.sum(new_comb*Power_key), N+1]), int(nk_data[comb, N+1])) not in edgelist:
                edgelist.append((int(nk_data[comb, N+1]), int(nk_data[np.sum(new_comb*Power_key), N+1])))

    node_attrs = {}
    for row in nk_data:
        node_attrs[int(row[N+1])] = {"fitness": row[N]}

    G.add_edges_from(edgelist)
    nx.set_node_attributes(G, node_attrs)

    num_edges_to_remove = int(edge_removal_percentage*(2**N)*N/2)
    edges_to_remove = random.choices(edgelist, k=num_edges_to_remove)
    G.remove_edges_from(edges_to_remove)

    extra_naming = name+'_' if name != "" else ""
    nx.write_gexf(G, '../data/landscape/'+extra_naming+'N'+str(N)+'K'+str(K)+'_'+str(int(edge_removal_percentage*100))+'.gexf')
```

Lastly, the `generate_landscape.py` file executes the actual creation of the NK landscape graph based on the provided arguments.

```{eval-rst}
.. automodule:: landscape.generate_landscape
   :members:
   :undoc-members:
   :show-inheritance:
```

(l.knowledge)=
## landscape.knowledge module

```{eval-rst}
.. automodule:: landscape.knowledge
   :members:
   :undoc-members:
   :show-inheritance:
```
### Knowledge sharing and aggregation

The `union(knowledgeList)` function is used to create a union of multiple knowledge objects represented as instances of a class called `Knowledge`. It operates on a list of knowledge objects and returns a new knowledge object that represents the union of the input knowledge objects.
* It initializes a new `knowledge_union` as a copy of the first knowledge object in the input list `knowledgeList` (i.e., `knowledge_union` = `Knowledge(knowledgeList[0])`). This will be the base for the union operation.
* It then iterates over the remaining knowledge objects in `knowledgeList` starting from the second one (index 1). For each of these knowledge objects:
   * It iterates over the landscape layers within the knowledge object. A layer is essentially a NetworkX-based graph structure.
   * It creates a reduced view of each layer (a subgraph) by removing nodes with zero degree (i.e., nodes that are not connected to any other nodes). This reduced view essentially filters out isolated nodes from the layers.
   * It uses the `nx.compose()` function from the NetworkX library to merge the reduced view of the layer with the corresponding layer in the `knowledge_union`. This operation combines the graphs from both knowledge objects, layer by layer.
   * If the layer is marked as active (according to the original knowledge object), it increments the `layerSharingNum` attribute in the `knowledge_union` for that layer. This attribute tracks the number of knowledge objects sharing a particular layer. 
* After the union process, it checks each layer in the `knowledge_union`. If a layer has at least one edge (i.e., it's not empty), it marks that layer as active in the `knowledge_union`. The purpose of this step is to identify active layers based on the merged information.
* Finally, it returns the `knowledge_union` as the result, which represents the union of knowledge from all the input knowledge objects. The resulting object contains information about shared layers and active layers, making it a consolidated representation of knowledge.
```python
def union(knowledgeList):
  knowledge_union = Knowledge(knowledgeList[0])
  for knowledge in knowledgeList[1:]:
    for l_idx, layer in enumerate(knowledge.layers):
      reduced_view = nx.subgraph_view(layer, filter_node=lambda n: layer.degree[n] > 0)
      knowledge_union.layers[l_idx] = nx.compose(knowledge_union.layers[l_idx], reduced_view)
      if knowledge.isLayerActive[l_idx]:
        knowledge_union.layerSharingNum[l_idx] += 1
  
  for l_idx, layer in enumerate(knowledge_union.layers):
    if layer.number_of_edges() > 0:
      knowledge_union.isLayerActive[l_idx] = True

  return knowledge_union
```

The `hasOverlap(knowledgeList)` function is used to determine whether there is an overlap in the nodes of specific layers among a list of knowledge objects. 
* It initializes a boolean variable `has_intersect` as `False`. This variable will be used to track whether there is an overlap.
* The function iterates over the layers of the first knowledge object in the knowledgeList. For each layer:
   * It initializes an intersect set with the nodes from that layer in the first knowledge object (i.e., `knowledgeList[0].nodesInLayers[l_idx]`).
   * It then iterates over the other knowledge objects (from the second one onwards) in `knowledgeList`. For each knowledge object:
   * It takes the intersection of the current intersect set with the nodes from the same layer in the current knowledge object (i.e., `knowledge.nodesInLayers[l_idx]`). This means it finds the common nodes shared by all knowledge objects for a specific layer.
   * If the size (length) of the intersect set is greater than 0, it means that there is an overlap in the nodes of that layer among all knowledge objects in `knowledgeList`. In that case, it sets the `has_intersect` variable to `True`.
* After iterating through all layers and checking for overlaps, the function returns the `has_intersect` boolean variable. 
```python
def hasOverlap(knowledgeList):
  has_intersect = False
  for l_idx, layer in enumerate(knowledgeList[0].layers):
    intersect = set(knowledgeList[0].nodesInLayers[l_idx])
    for knowledge in knowledgeList[1:]:
      intersect = intersect.intersection(knowledge.nodesInLayers[l_idx])
    if len(intersect) > 0:
      has_intersect = True

  return has_intersect
```

The `getShareableKnowledge(sourceKnowledge, targetKnowledge, isLayerRevealAllowed)` function is designed to identify and return a set of edges that can be shared between two individuals (source and target) in each available layer within their respective knowledge graphs. The function considers whether layer reveal is allowed and takes the following steps to determine shareable edges:
* It initializes an empty list `shareable_edges` to store the edges that can be shared between the source and target individuals.
* The function iterates over each layer in the knowledge graphs. For each layer (indexed by `l_idx`), it does the following:
   * It calculates the intersection of nodes between the source and target knowledge graphs for the current layer. This identifies nodes that both individuals have in common in that layer.
   * For each node in the intersection, the function then iterates over the edges in the source knowledge graph for that node. It forms an edge with the corresponding layer index and checks whether the edge is already in the `shareable_edges` list and whether it is missing in the target knowledge graph.
   * If the edge meets these conditions, it is added to the `shareable_edges` list, indicating that it can be shared between the source and target individuals.
* Finally, the function returns the `shareable_edges` list, which contains information about the edges that can be shared between the source and target individuals in each available layer.
```python
def getShareableKnowledge(sourceKnowledge, targetKnowledge, isLayerRevealAllowed):
  shareable_edges = []
  for l_idx in range(len(sourceKnowledge.layers)):
    intersection = set(sourceKnowledge.nodesInLayers[l_idx]).intersection(targetKnowledge.nodesInLayers[l_idx])
    for node in intersection:
      for edge in sourceKnowledge.layers[l_idx].edges(node):
        edge_l = [edge, l_idx]
        if (edge_l not in shareable_edges) and not targetKnowledge.layers[l_idx].has_edge(*edge):
          shareable_edges.append(edge_l)

  return shareable_edges
```
### Knowledge object (instance) creation

The `Knowledge` class is designed to represent and alter a knowledge object, which includes multiple layers of knowledge, nodes within those layers, and learned edges between nodes. It can be initialized in two different ways, as indicated by the `__init__` constructor.

1. Initialization of a `Knowledge` object as a subgraph of a landscape (multiple arguments provided):
* If the constructor receives more than one argument, it is initialized from a set of parameters (e.g. landscape, sourceNode etc.).
* The constructor then initializes the following attributes:
   * `landscape`: Stores the associated landscape or graph structure.
   * `capacity`: Stores the capacity value.
   * `layers`: A list that will store graph structures for each layer.
   * `nodesInLayers`: A list that will store the nodes within each layer.
   * `isLayerActive`: A list that tracks the activation status of each layer.
   * `learned_edges`: A deque that stores learned edges. A deque is a double-ended queue in which elements can be both inserted and deleted from either the left or the right end of the queue.
* For each layer specified in `initLayers` (argument), it initializes a graph `G` for that layer and activates the layer. It then proceeds to populate the layer:
   * A node from the `nodes_to_consider` list is randomly selected.
   * It checks if the degree of the selected node in `G` is less than or equal to `initMaxDegree` argument.
   * If the condition is met, it selects a neighboring node from the landscape, connects it to the selected node in `G`, and adds the edge to the `learned_edges` deque.
   * The selected neighboring node is added to the `nodes_to_consider` list.
   This process continues until `initNodeNum` nodes are initialized in the layer or the loop exceeds a maximum number of iterations.
* The constructed graph `G` for each layer is stored in the `layers` list, and the nodes within each layer are stored in the `nodesInLayers` list.

2. Initialization of knowledge overlap implementation (only the landscape provided as an argument):
* If the constructor receives a single argument, it is initialized from an existing knowledge graph, which is useful for copying or extending existing knowledge.
* The existing knowledge graph is provided as the argument.
* The constructor copies various attributes from the existing knowledge, including the associated landscape, layer activation status, layer sharing information, nodes within layers, capacity, and learned edges. It also copies the graph structures for each layer.
```python
class Knowledge:
  def __init__(self, *args):
    if len(args) > 1:
      landscape = args[0]
      sourceNode = args[1]
      initLayers = args[2]
      initNodeNum = args[3]
      initMaxDegree = args[4]
      capacity = args[5]

      self.landscape = landscape
      self.capacity = capacity
      self.layers = []
      self.nodesInLayers = []
      self.isLayerActive = []
      self.learned_edges = deque()

      self.layerSharingNum = []

      for l_idx in range(len(self.landscape.layers)):
        G = nx.create_empty_copy(self.landscape.layers[l_idx])
        self.isLayerActive.append(False)
        self.layerSharingNum.append(0)
        if l_idx in initLayers:
          self.isLayerActive[l_idx] = True
          self.layerSharingNum[l_idx] += 1
          iter_num = 1
          node_num = 1
          nodes_to_consider = [sourceNode]
          while node_num <= initNodeNum and iter_num < initNodeNum*10:
            iter_num += 1
            selected_node = np.random.choice(nodes_to_consider)
            if G.degree[selected_node] <= initMaxDegree:
              neighbours = [n for n in self.landscape.layers[l_idx].neighbors(selected_node)]
              new_node = np.random.choice(neighbours)
              if not G.has_edge(selected_node, new_node):
                G.add_edge(selected_node, new_node)
                self.learned_edges.append((selected_node, new_node,))
                if new_node not in nodes_to_consider:
                  nodes_to_consider.append(new_node)
                  node_num += 1
          
        self.layers.append(G)
        reduced_view = nx.subgraph_view(G, filter_node=lambda n: G.degree[n] > 0)
        self.nodesInLayers.append(list(reduced_view.nodes))

    else:
      knowledge = args[0]
      self.landscape = knowledge.landscape
      self.layers = []
      self.isLayerActive = knowledge.isLayerActive
      self.layerSharingNum = knowledge.layerSharingNum
      self.nodesInLayers = knowledge.nodesInLayers
      self.capacity = knowledge.capacity
      self.learned_edges = knowledge.learned_edges

      for layer in knowledge.layers:
        self.layers.append(layer.copy())

```
The `addEdge(self, edge, layer)` method of the `Knowledge` class is used to add an edge to a specific layer within the knowledge object. 
* It takes two arguments:
   * `edge`: The edge to be added, which is typically represented as a tuple of two nodes.
   * `layer`: The index of the layer to which the edge should be added.
* It adds the specified edge to the graph structure of the specified layer using the `add_edge` method of the NetworkX graph (`self.layers[layer].add_edge(*edge)`).
* It sets the `isLayerActive` flag for the specified layer to `True`. This indicates that the layer is active because it contains at least one edge.
* It appends the edge to the `learned_edges` deque. This deque is used to keep track of the learned edges in the knowledge object.
* It checks if the number of edges in the graph of the specified layer exceeds the defined capacity (`self.capacity`). If the capacity is exceeded, the method removes the oldest learned edge from the layer.
* It updates the `nodesInLayers` attribute for the specified layer. It creates a reduced view of the graph of that layer, filtering out nodes with zero degree, and updates the list of nodes within that layer based on the reduced view.  
```python
class Knowledge: CONTINUED
  def addEdge(self, edge, layer):
    self.layers[layer].add_edge(*edge)
    self.isLayerActive[layer] = True
    self.learned_edges.append(edge)
    if self.layers[layer].number_of_edges() > self.capacity: 
      self.layers[layer].remove_edge(*self.learned_edges.popleft())
    reduced_view = nx.subgraph_view(self.layers[layer], filter_node=lambda n: self.layers[layer].degree[n] > 0)
    self.nodesInLayers[layer] = reduced_view.nodes
```
The `distance` function takes a `sourceNode` and a `targetNode`, merges the edges from multiple layers of a knowledge structure to form a flattened multiplex graph, and then calculates and returns the shortest path length between the source and target nodes within this merged graph.
* It starts by initializing a new graph, `flattened_multiplex_graph`, by making a copy of the first layer of the knowledge structure (`self.layers[0].copy()`). This graph represents a merged version of the multiplex graph, combining edges from multiple layers.
* It then iterates over the remaining layers of the knowledge structure and updates the `flattened_multiplex_graph` by adding the edges from each of these layers to the graph. The `.update()` method is used to combine the edges of the current layer with the existing graph.
* After merging all the layers, the function calculates the shortest path length between the `sourceNode` and the `targetNode` within the `flattened_multiplex_graph` using the `nx.shortest_path_length()` function from the NetworkX library.
* Finally, the function returns the calculated shortest path length as the result.
``` python
class Knowledge: CONTINUED
  def distance(self, sourceNode, targetNode):
    flattened_multiplex_graph = self.layers[0].copy()
    for layer in self.layers[1:]:
      flattened_multiplex_graph.update(edges=layer.edges)

    return nx.shortest_path_length(flattened_multiplex_graph, sourceNode, targetNode)
```
The `unknownNeighbors` method is designed to identify and return the neighbors of a `sourceNode` in a specific layer of the knowledge object that are unknown (i.e. not connected) in the same layer of the knowledge graph.
* It starts by obtaining a list of all neighbors of the `sourceNode` within the specified layer of the associated landscape.
* It then obtains a list of neighbors that are known or connected in the same layer of the knowledge graph (`known_neighbours`). These are retrieved from the graph structure of the specified layer in the knowledge object.
* Next, it initializes an empty list `unknown_neighbours` to store the neighbors that are unknown or not connected in the knowledge graph.
* It iterates through each node in the `all_neighbours` list and checks if the node is also present in the `known_neighbours` list. If a node is not found in the known_neighbours list, it means that the node is unknown or not connected in the same layer of the knowledge graph. In this case, the node is added to the `unknown_neighbours` list.
* Finally, the method returns the list of `unknown_neighbours`.
```python
class Knowledge: CONTINUED
  def unknownNeighbors(self, sourceNode, layer):
    all_neighbours = [n for n in self.landscape.layers[layer].neighbors(sourceNode)]
    known_neighbours = [n for n in self.layers[layer].neighbors(sourceNode)]
    unknown_neighbours = []
    for node in all_neighbours:
      if node in known_neighbours:
        unknown_neighbours.append(node)

    return unknown_neighbours
```
### Knowledge graph visualisation

The `visualise` method is used to visualise the knowledge graph. It provides options to customize the visualisation.
* It starts by creating an empty list `reduced_layers`, which will store copies of the graph structures for layers that are active. It iterates through each layer in and appends a copy of the layer to `reduced_layers` if it is active.
* If `includeUnreachable` is set to `False` (the default), the method performs additional filtering to remove nodes that are unreachable in all layers. This ensures that only nodes that have at least one connection within the knowledge structure are included in the visualisation.
   * It iterates through nodes in the first layer, and for each node, it checks if the node is unreachable in all layers. If the node is unreachable in all layers, it is marked for removal.
   * After identifying nodes to remove, it iterates through `reduced_layers` and removes the marked nodes from each layer.
* It initializes a 3D figure `fig` and adds a 3D subplot `ax` to it using matplotlib.
* It creates the network visualisation using a custom function called `MultiplexNetworkVisualisation` (**refer to visualisation.py**). The `reduced_layers` are passed to this function, and the visualisation is generated on the ax subplot. 
* It turns off the axis for the 3D subplot using `ax.set_axis_off()` to make the visualisation cleaner.
* If `disableShow` is set to False (the default), it displays the visualisation using `plt.show()`. If disableShow is set to True, it will not automatically display the visualisation.
```python
class Knowledge: CONTINUED
  def visualise(self, includeUnreachable=False, disableShow=False, nodeSize=80):
    reduced_layers = []
    for l_idx, layer in enumerate(self.layers):
      if self.isLayerActive[l_idx]:
        reduced_layers.append(layer.copy())

    if not includeUnreachable:
      for node in self.layers[0].nodes():
        remove_node = True
        for layer in self.layers:
          if layer.degree[node] > 0:
            remove_node = False

        if remove_node:    
          for layer in reduced_layers:
            layer.remove_node(node)

    fig = plt.figure()
    ax = fig.add_subplot(111, projection='3d')
    MultiplexNetworkVisualisation(reduced_layers, ax=ax, layout=nx.spring_layout, node_size=nodeSize)
    ax.set_axis_off()
    if not disableShow:
      plt.show()
```

## landscape.make_multiplex_landscape module

```{eval-rst}
.. automodule:: landscape.make_multiplex_landscape
   :members:
   :undoc-members:
   :show-inheritance:
```

### Multiplex landscape construction

It starts by importing the necessary modules, including networkx for working with graphs and sys for handling command-line arguments.
```python
import sys
import networkx as nx
```

 The `make_multiplex_landscape` function defines the construction of a multiplex network (a network with multiple layers) from a list of graphs, associates fitness values with nodes, and writes the resulting multiplex network to a .gexf file for further analysis and visualization. This function takes a name and a list of graphs as input.
 * It initializes a MultiGraph named `G_mpx`. A MultiGraph is a type of graph structure in NetworkX that allows multiple edges (parallel edges) between the same pair of nodes. This structure is suitable for creating a multiplex network where each layer is represented as a separate edge set.
 * It adds nodes to `G_mpx` from the first graph in the list of graphs (i.e., graphs[0]). The nodes in `G_mpx` are initially populated from the nodes in the first graph.
 * It then iterates through the list of graphs. For each graph in the list, it performs the following actions:
   * It iterates over the nodes of the current graph and assigns a node attribute to each node in G_mpx. The node attribute is named after the index of the current graph (i.e., i) and is set to the fitness value of the corresponding node in the current graph. This allows fitness values from different layers (graphs) to be associated with the same node in the multiplex network.
   * It adds edges to `G_mpx` from the current graph with the specified layer index (i). These edges represent connections or relationships within a specific layer of the multiplex network.
* After iterating through all the graphs and adding nodes and edges to G_mpx, the function writes the resulting multiplex network to a GEXF (Graph Exchange XML Format) file in the '../data/landscape/' directory. The file is named using the provided name with a '.gexf' extension.
 ```python
 def make_multiplex_landscape(name, graphs):
    G_mpx = nx.MultiGraph()
    G_mpx.add_nodes_from(graphs[0])
    for i, graph in enumerate(graphs):
        for k in range(0, len(graph.nodes)):
            G_mpx.nodes[k][i] = graph.nodes[k]["fitness"]
        G_mpx.add_edges_from(graph.edges, layer=i)

    nx.write_gexf(G_mpx, '../data/landscape/'+name+'.gexf')
 ```

Next, a script is designed to process a list of GEXF files representing individual layers of a multiplex network and create a multiplex network from these layers, with the ability to assign fitness values to nodes in each layer. The resulting multiplex network is saved to a GEXF file for further analysis and visualization. The script takes the desired name for the output file and a list of input files as command-line arguments.
* It initializes an empty list called `graphs`.
* The script uses a for loop to iterate through the command-line arguments (`sys.argv`). The loop starts from the third argument (index 2) and continues to the end of the arguments. The reason for starting at index 2 is that the first argument (sys.argv[0]) is the script's name, and the second argument (sys.argv[1]) is expected to be the desired name for the output file.
* Within the loop, it reads GEXF files specified in the command-line arguments using `nx.read_gexf()`. Each GEXF file represents a separate graph or layer of the multiplex network. The loaded graphs are added to the graphs list.
* After all the specified GEXF files have been read and the corresponding graphs are stored in the graphs list, the script calls the `make_multiplex_landscape` function with two arguments:
   * The first argument (`sys.argv[1]`) is expected to be the desired name for the output file.
   * The second argument is the list of loaded graphs (`graphs`).
* The `make_multiplex_landscape` function constructs a multiplex network from the list of graphs and writes the resulting multiplex network to a GEXF file with the specified name.
```python
graphs = []
for i in range(2, len(sys.argv)):
    graphs.append(nx.read_gexf('../data/landscape/'+sys.argv[i]+'.gexf', node_type=int))

make_multiplex_landscape(sys.argv[1], graphs)
```
(l.multiplex)=
## landscape.multiplex module

```{eval-rst}
.. automodule:: landscape.multiplex
   :members:
   :undoc-members:
   :show-inheritance:
```

This module primarily serves to efficiently assess shortest path lengths in a pregenerated multiplex landscape and fitness values of the associated nodes in the network.
### Library and module imports

First, some necessary libraries and modules are imported, including networkx for network analysis, matplotlib for plotting, and `MultiplexNetworkVisualisation` as constructed in the [landscape.visualisation module](#l.v). 
```python
import networkx as nx
import matplotlib.pyplot as plt
from landscape.visualisation import MultiplexNetworkVisualisation
```

### Shortest pathway object initialisation

The `MultiplexLandscape` class represents a multiplex network landscape by reading network layers from GEXF files and precomputing the shortest path lengths between nodes in each layer. This allows for efficient analysis and manipulation of the multiplex network landscape.
* `self.num_layers` is initialised with the number of layers, which is the length of the `file_list` (argument).
* `self.layers` is initialised as an empty list.
* A for loop iterates through the file names in `file_list`, and for each file, it reads the GEXF file located in the specified path ('./data/landscape/') using nx.read_gexf. The resulting network is added to the `self.layers` list.
* `self.path_length` is initialised as a list, where each element of the list corresponds to a network layer. For each layer, it computes and stores the shortest path lengths between all pairs of nodes using `nx.all_pairs_shortest_path_length`.
```python
class MultiplexLandscape: 
  def __init__(self, file_list):
    self.num_layers = len(file_list)
    self.layers = []
    for filename in file_list:
        self.layers.append(nx.read_gexf('./data/landscape/'+filename+'.gexf', node_type=int))
    
    self.path_length = [dict(nx.all_pairs_shortest_path_length(layer)) for layer in self.layers]
```

### Retrieving fitness values

The `MultiplexLandscape` class provides two methods for retrieving fitness values associated with nodes in the network layers. These methods allow to access fitness values for specific nodes within individual layers or across all layers of the multiplex network landscape.
* `getFitnessForLayer(self, layer, node)`: This method retrieves the fitness value of the specified node in the specified layer and returns it. It takes two arguments:
   * `layer`: An integer representing the index of the layer within the multiplex landscape.
   * `node`: An integer representing the node for which to retrieve the fitness value in the specified layer.
* `getFitness(self, node)`: This method iterates through all layers and retrieves the fitness value associated with the specified node in each layer. It returns a list of fitness values, where each element of the list corresponds to a different layer in the multiplex landscape. It takes a single argument:
   * `node`: An integer representing the node for which to retrieve fitness values across all layers.
```python
class MultiplexLandscape: CONTINUED
  def getFitnessForLayer(self, layer, node):
    nodes = self.layers[layer].nodes(data=True)
    fitness = nodes[node]["fitness"]
    return fitness

  def getFitness(self, node):
    fitness_list = []
    for l_idx in range(self.num_layers):
      nodes = self.layers[l_idx].nodes(data=True)
      fitness_list.append(nodes[node]["fitness"])

    return fitness_list
```

### Visualisation

Lastly, the `visualise` method in the `MultiplexLandscape` class is used to visualize the multiplex landscape using a 3D plot. It allows for creating a 3D visualization of the multiplex landscape, and its custimisation using parameters control whether it's displayed interactively.
* It creates a new 3D plot within a figure.
* It uses the `MultiplexNetworkVisualisation` class to visualise the multiplex landscape, passing in the layers of the landscape as input data. The layout parameter is set to `nx.spring_layout`, which is a NetworkX layout algorithm used to determine the positions of nodes in the visualisation.
* It sets the size of the nodes in the visualisation using the `node_size` parameter.
* It turns off the axis labels using `ax.set_axis_off()`, making the plot cleaner.
* Finally, it shows the plot using `plt.show()` unless the `disableShow` parameter is set to True. If `disableShow` is True, the plot won't be displayed, which can be useful for saving the plot as an image file or use it for further analysis without displaying it interactively.
```python
  def visualise(self, disableShow=True, node_size=300):
    fig = plt.figure()
    ax = fig.add_subplot(111, projection='3d')
    MultiplexNetworkVisualisation(self.layers, ax=ax, layout=nx.spring_layout, node_size=node_size)
    ax.set_axis_off()
    if not disableShow:
      plt.show()
```

(l.v)=
## landscape.visualisation module

```{eval-rst}
.. automodule:: landscape.visualisation
   :members:
   :undoc-members:
   :show-inheritance:
```

### Library and module imports

```python
import numpy as np
import matplotlib.pyplot as plt
import networkx as nx

from mpl_toolkits.mplot3d import Axes3D
from mpl_toolkits.mplot3d.art3d import Line3DCollection
```

### Initialisation of 3D-multiplex network visualisation

A class named `MultiplexNetworkVisualisation` is defined for the purpose of visualising multiplex networks in 3D space.
* The class constructor (`__init__`) takes several parameters to set up the visualisation:
   * `graphs`: A list of `networkx.Graph` objects, one for each layer of the multiplex network.
   * `node_labels`: A dictionary mapping node IDs to labels (or None if nodes should not be labeled).
   * `layout`: A layout function (defaulting to `networkx.spring_layout`) is used to compute the positions of nodes.
   * `ax`: An optional instance of `mpl_toolkits.mplot3d.Axes3d` where the visualization will be plotted. If not provided, a new figure and axis are created.
* The constructor stores the input parameters in instance variables for later use, such as `self.graphs`, `self.node_labels`, `self.layout`, and `self.node_size`
* Since an `ax` parameter is not provided, a new 3D axis is created using matplotlib.
* The constructor then creates an internal representation of nodes and edges for the multiplex network, considering connections within and between layers.
   * `self.get_nodes()`: This method compiles a list of all nodes present in any of the layers of the multiplex network.
   * `self.get_edges_within_layers()`: This method determines the edges within each layer by iterating through the provided graphs.
   * `self.get_edges_between_layers()`: This method determines the edges between layers. Nodes in subsequent layers are connected if they have the same node ID. This allows the visualisation to show connections between the same nodes across different layers.
* The constructor calculates the positions of nodes in 3D space using the `self.get_node_positions` function.
* The `self.draw()` method is then called to create the actual visualisation based on the computed node positions, edges within layers, and edges between layers. This visualisation represents the multiplex network with different layers separated along the z-axis. The provided `node_size` parameter controls the size of the nodes in the visualisation.
```python
class MultiplexNetworkVisualisation(object):
    def __init__(self, graphs, node_labels=None, layout=nx.spring_layout, ax=None, node_size=300):
        self.graphs = graphs
        self.total_layers = len(graphs)

        self.node_labels = node_labels
        self.layout = layout

        self.node_size = node_size

        if ax:
            self.ax = ax
        else:
            fig = plt.figure()
            self.ax = fig.add_subplot(111, projection='3d')

        self.get_nodes()
        self.get_edges_within_layers()
        self.get_edges_between_layers()

        self.get_node_positions()
        self.draw()
```

### Class method definitions

Several of the functions which the `__init__` constructors calls are defined in the following code section.
* `get_nodes(self)`: This method constructs an internal representation of nodes in the multiplex network. The internal representation is a list of tuples, where each tuple contains a node ID and its corresponding layer. The method iterates through the list of graphs (one for each layer) and collects nodes from each layer, mapping each node to its layer index.
* `get_edges_within_layers(self)`: This method remaps edges within individual layers to the internal representations of node IDs. It creates a list of tuples, where each tuple contains a pair of nodes in the format ((source node, layer), (target node, layer)). It iterates through the list of graphs and their edges, mapping each edge to the internal representation with layer information.
* `get_edges_between_layers(self)`: This method determines the edges between layers in the multiplex network. It assumes that nodes in subsequent layers are connected if they have the same ID. It creates a list of tuples, where each tuple contains a pair of nodes in different layers that share the same node ID. It iterates through pairs of adjacent graphs (layers) and identifies nodes that exist in both layers, indicating connectivity between layers.
```python
class MultiplexNetworkVisualisation(object): CONTINUED
    def get_nodes(self):
        self.nodes = []
        for z, g in enumerate(self.graphs):
            self.nodes.extend([(node, z) for node in g.nodes()])


    def get_edges_within_layers(self):
        self.edges_within_layers = []
        for z, g in enumerate(self.graphs):
            self.edges_within_layers.extend([((source, z), (target, z)) for source, target in g.edges()])


    def get_edges_between_layers(self):
        self.edges_between_layers = []
        for z1, g in enumerate(self.graphs[:-1]):
            z2 = z1 + 1
            h = self.graphs[z2]
            shared_nodes = set(g.nodes()) & set(h.nodes())
            self.edges_between_layers.extend([((node, z1), (node, z2)) for node in shared_nodes])
```

Also, the code defines a method called `get_node_positions`. This method is used to compute the positions of nodes in 3D space for the multiplex network visualisation.
* The method starts by initialising a variable called `composition` with the first graph from the list of `self.graphs`. This graph is referred to as the composition because it represents the composition of all layers of the multiplex network.
* The method then iterates through the remaining graphs (layers) in `self.graphs` starting from the second graph.
* For each layer, it combines the composition graph with the current layer `h` using the `nx.compose` function from NetworkX. This results in a single graph that represents the entire multiplex network with all layers.
* Next, the method computes the node positions in 3D space using a specified layout function (`self.layout`). It passes the composed graph composition as well as any additional arguments and keyword arguments provided in `*args` and `**kwargs`.
* An empty dictionary called `self.node_positions` is created, which will store the positions of nodes in the format (node ID, layer) : (x, y, z, fitness).
* The method then iterates through the list of graphs (layers) in `self.graphs`, and for each graph, it associates node positions with layers.
* For each node in a graph (layer), the method adds an entry to the `self.node_positions` dictionary. The entry's key is a tuple containing the node's ID and layer, and the value is a tuple containing the node's 3D position (x, y, z) and its fitness value. The result is a `self.node_positions` dictionary that maps each node in the multiplex network to its 3D position and fitness value.
```python
class MultiplexNetworkVisualisation(object): CONTINUED
    def get_node_positions(self, *args, **kwargs):
        composition = self.graphs[0]
        for h in self.graphs[1:]:
            composition = nx.compose(composition, h)

        pos = self.layout(composition, *args, **kwargs)

        self.node_positions = dict()
        for z, g in enumerate(self.graphs):
            self.node_positions.update({(node[0], z) : (*pos[node[0]], z, float(node[1]['fitness'])) for node in g.nodes(data=True)})
```

### Draw methods

The `draw_nodes` method is responsible for rendering and visualising the nodes of the multiplex network in a 3D plot. The result is a 3D scatter plot of the specified nodes in the multiplex network, where the color of each node is determined by its fitness value, and the nodes are distributed in 3D space according to their positions.
```python
class MultiplexNetworkVisualisation(object): CONTINUED
    def draw_nodes(self, nodes, *args, **kwargs):
        x, y, z, f = zip(*[self.node_positions[node] for node in nodes])
        self.ax.scatter(x, y, z, c=f, cmap='Purples', *args, **kwargs)
```

The `draw_edges` method is responsible for rendering and visualising the edges (connections) between nodes in the multiplex network in a 3D plot. The result is a 3D plot that visualises the edges (connections) between nodes in the multiplex network. The line segments connecting the nodes are rendered in 3D space according to their starting and ending coordinates. 
```python
class MultiplexNetworkVisualisation(object): CONTINUED
    def draw_edges(self, edges, *args, **kwargs):
        segments = [(
            (self.node_positions[source][0], self.node_positions[source][1], self.node_positions[source][2]), 
            (self.node_positions[target][0], self.node_positions[target][1], self.node_positions[target][2])
        ) for source, target in edges]
        line_collection = Line3DCollection(segments, *args, **kwargs)
        self.ax.add_collection3d(line_collection)
```

The `get_extent` method is responsible for calculating the extent of the 3D plot and setting the boundaries of the plot when rendering it, taking into account the positions of nodes.
```python
class MultiplexNetworkVisualisation(object): CONTINUED
   def get_extent(self, pad=0.1):
        xyz = np.array(list(self.node_positions.values()))
        xmin, ymin, _, _ = np.min(xyz, axis=0)
        xmax, ymax, _, _ = np.max(xyz, axis=0)
        dx = xmax - xmin
        dy = ymax - ymin
        return (xmin - pad * dx, xmax + pad * dx), \
            (ymin - pad * dy, ymax + pad * dy)
```

The `draw_plane` method is responsible for drawing a plane (i.e. a flat surface) at a specified `z` height position within the 3D plot, covering the extent of the x-y plane.
```python
class MultiplexNetworkVisualisation(object): CONTINUED
   def draw_plane(self, z, *args, **kwargs):
        (xmin, xmax), (ymin, ymax) = self.get_extent(pad=0.1)
        u = np.linspace(xmin, xmax, 10)
        v = np.linspace(ymin, ymax, 10)
        U, V = np.meshgrid(u ,v)
        W = z * np.ones_like(U)
        self.ax.plot_surface(U, V, W, *args, **kwargs)
```
The `draw_node_labels` method iterates through the nodes in the multiplex network, checks if each node has a label, and if so, it adds a label at the 3D position of the node in the plot. The appearance of the labels can be customised using the provided arguments and keyword arguments.
```python
class MultiplexNetworkVisualisation(object): CONTINUED
   def draw_node_labels(self, node_labels, *args, **kwargs):
        for node, z in self.nodes:
            if node in node_labels:
                ax.text(*self.node_positions[(node, z)], node_labels[node], *args, **kwargs)
```

Lastly, the `draw` method is responsible for drawing the multiplex network in 3D. It assembles and visualises the multiplex network by drawing edges, planes representing layers, nodes, and labels. It allows for customisation of various visual aspects, such as colors, transparency, and sizes, to create a 3D representation of the network.
```python
class MultiplexNetworkVisualisation(object): CONTINUED
   def draw(self):

        self.draw_edges(self.edges_within_layers,  color='k', alpha=0.3, linestyle='-', zorder=2)
        self.draw_edges(self.edges_between_layers, color='k', alpha=0.3, linestyle='--', zorder=2)

        for z in range(self.total_layers):
            self.draw_plane(z, alpha=0.2, zorder=1)
            self.draw_nodes([node for node in self.nodes if node[1]==z], s=self.node_size, zorder=3)

        if self.node_labels:
            self.draw_node_labels(self.node_labels,
                                  horizontalalignment='center',
                                  verticalalignment='center',
                                  zorder=100)
```
## Module contents

```{eval-rst}
.. automodule:: landscape
   :members:
   :undoc-members:
   :show-inheritance:
```