Anatomical Flatmap Resources
Flatmap resources
Mapmaker
Mapmaker is a Python application that generates Mapbox compatible tilesets from a range of sources (currently including SVG drawings, MBF Biosciences segmented images, and PowerPoint slides). In conjunction with SCKAN, Mapmaker is used to render the flatmaps portraying the anatomy and nerve knowledge of species, as seen on the SPARC Portal Maps page.
Flatmaps can be generated and viewed by using the flatmap maker (mapmaker), server, and viewer. For more information on these flatmap resources see the SPARC Map Reuse documentation.
SPARC Connectivity Knowledge Base of the Autonomic Nervous System (SCKAN)
SCKAN is a key component of the SPARC Program, providing comprehensive knowledge of the autonomic nervous system (ANS) nerve to end organ connectivity. Derived from SPARC experts, literature and textbooks, SCKAN provides the resulting connectivity seen on the SPARC Portal flatmaps. For more information on SCKAN, refer to the documentation below.
ApiNATOMY
SCKAN knowledge is based on various ApiNATOMY models set up by SPARC investigators. For more information on ApiNATOMY, visit the SPARC SAWG site. The ApiNATOMY models can be accessed via the github repository.
Simply clone the repository.
$ git clone https://github.com/open-physiology/apinatomy-models
Navigate to the desired model in the models folder. Each model can be found in the docs folder, in the form of .svg and .pdf files.
Map-tools
Map-tools is a jupyter notebook package that provides information on individual neuron connections, according to the knowledge in SCKAN. It can be used to understand each connection and the respective anatomical features involved.
Installation
Map-tools is best run under a python environment and is reliant on Poetry. Map-tools can then be installed by cloning the GitHub repository.
$ git clone https://github.com/AnatomicMaps/map-tools
Having installed Poetry via the official installer instructions, clone and enter the map-tools directory. Install a Poetry environment with
$ poetry install
Usage
Once installed, map-tools can be run by opening Jupyter Notebook. In the map-tools directory, run jupyter notebook via poetry.
$ poetry run jupyter notebook
The command will provide access to the notebook with a URL, which should then be pasted into a browser window.
Once opened, navigate to and open the connectivity_graph.ipynb notebook in the connectivity-graph folder. A valid SCICRUNCH Key must be input in order to draw knowledge from SCKAN. This can be done by inserting the key under a scicrunch_key field as shown in Figure 2. Note that a SciCrunch key can be generated by setting up a SciCrunch account.
Neuron paths from ApiNATOMY Models can now be visualised, as seen in Figure 3. The desired neuron path can be visualised by changing the entity field. Table 1 represents the ilxtr attribute for each model. Note that the ID of each path is dependent on the visualisation of paths in each model.
Model | ilxtr (‘X’ represents path ID) |
---|---|
ApiNATOMY model of bronchomotor control | neuron-type-bromo-X |
ApiNATOMY model of the spleen | neuron-type-splen-X |
ApiNATOMY model of the stomach | neuron-type-sstom-X |
Ardell-Armour model of the heart | neuron-type-aacar-X |
Bolser-Lewis model of defensive breathing | neuron-type-bolew-unbranched-X |
Keast model of the bladder | neuron-type-keast-X |
SAWG model of the distal colon | neuron-type-sdcol-X |
Table 1: ilxtr attributes of each path in the respective ApiNATOMY models. These are used to visualise said paths in the connectivity_graphy.ipynb file.
Mapmaker Installation
It is recommended to install and run mapmaker under its own Python virtual environment. Mapmaker can be installed as a python application, or a development version can be set up. The Installing mapmaker in a development environment section explores how to set up the development version of mapmaker. Further instructions for each OS and/or environment can be found in the Installation for various environments section.
The mapmaker resources include flatmap-maker, flatmap-server, and flatmap-viewer. Refer to the SPARC Map Reuse documentation for more information. Each repository contains documentation regarding installation.
https://github.com/AnatomicMaps/flatmap-maker
https://github.com/AnatomicMaps/flatmap-server
https://github.com/AnatomicMaps/flatmap-viewer
Installing and using the mapmaker application
Simply install the mapmaker wheel and the flatmap server by following the respective instructions. For more information on how to install Python wheels, click here. Run mapmaker with the appropriate command, and start the flatmap server to view the map.
Refer to the documentation for more information:
Note:
Be sure to use the latest release of mapmaker when installing the .whl file. Figure 4 highlights where in the flatmap maker repository to find the mapmaker releases.
Installing mapmaker in a development environment
Prerequisites
The following are required in order to successfully install and generate maps. Links to the packages have been provided, which capture installation instructions.
- Mapmaker:
- Python 3.10
- Tippecanoe
- COIN-OR CBC Solver
- Poetry
- Miniforge (for ARM based MacOS machines)
- Flatmap viewer
- Flatmap server
- Python 3.8 (or higher)
- Poetry
- Libgl1-mesa-glx
Installation instructions
Note:
Examples, including specific commands, for Windows and MacOS can be found in the Installation for various environments section.
The first step is to install the prerequisite packages.
-
- make
- g++
- sqlite libsqlite3-dev
- libz-dev
- coinor-cbc
- Set up the Tippecanoe package, following the official instructions.
Note:
Several warnings may appear but can be neglected. However, error messages in red may indicate failure.
- In the root directory, clone the flatmap-maker, -server, and -viewer repositories.
Note:
This will clone the head of the main branch of each repository. It is recommended to use the latest released version of mapmaker. Switch to the latest release while in the flatmap-maker directory (shown in Figure 4).
- As aforementioned, the flatmap resources are reliant on Poetry. Following the link, install Poetry via the instructions provided.
Note:
The ‘poetry’ command may not be recognised. This can be resolved by adding Poetry to the PATH, as per the official installation instructions.
- Once this has been done, enter the flatmap-maker directory and set up a Poetry environment. Test the installation by opening a poetry environment.
- Exit the poetry environment. In the flatmap-server directory, set up another Poetry environment. The mapserver can be started with
$ poetry run mapserver
Note:
A warning may appear relating to an undefined SCRICRUNCH_API_KEY. This can be ignored as the SciCrunch key will be added via mapmaker.
- Enter the flatmap-viewer directory and install npm. The output of npm install may mention several vulnerabilities, which can be resolved by following the suggested fix.
Note:
As mentioned in the Prerequisites section, node.js must be installed for the flatmap viewer. Click the link provided in the Prerequisites section and follow the instructions.
The flatmap viewer can be started with
$ npm start
Note:
The flatmap viewer may fail to start due to the updated version of Nodejs. This can be resolved by using Node Version Manager to install v12.22.9._
- The flatmap resources have been installed and are ready for use. Flatmaps will require a source, including a set of JSON files and an SVG file (see Flatmap sources for more information). For an up-to-date version of the rat flatmap (to be used alongside the developer environment of mapmaker), the repository below should be cloned. The set of sources for the flatmaps of other species found on the SPARC Portal can be found within the AnatomicMaps organization.
https://github.com/AnatomicMaps/rat-flatmap
Note:
This link is for the latest flatmap resources not an official release.
Flatmaps should now be ready for generation.
Using mapmaker
Once the flatmap sources have been appropriately created, mapmaker can now generate the flatmap. Mapmaker has various arguments that can be used to generate maps. These arguments, some optional, can allow for different visualizations of the flatmaps, and for debugging purposes.
Mapmaker involves various required and optional arguments (refer to the documentation for more information). Table 2 contains the required arguments to generate a map.
--source PATH TO SOURCE MANIFEST | Points mapmaker to the source of flatmap (i.e. the manifest.json) |
--output PATH TO DIRECTORY | Points mapmaker to the directory of the base of the generated flatmap |
Table 2: The required arguments when running mapmaker. ‘--source’ refers to the path to the directory of the source files’ manifest.json. ‘--output’ should point to the directory to dump the knowledgebase.db file.
- Enter the flatmap-maker directory and spawn a poetry shell.
- The flatmaps utilise the SciCrunch knowledgebase, to use this service a valid SciCrunch API Key must be set. A SciCrunch key can be obtained from the SciCrunch website, following these instructions.
$ export SCICRUNCH_API_KEY=*Enter a valid SciCrunch Key*
- Run mapmaker using the following command (with the appropriate and any desired arguments).
$ python runmaker.py --source _Enter path to flatmap source_ --output _Enter path to output directory_
Given mapmaker runs with no crashes or issues, the flatmap has now been successfully generated. The flatmap can be viewed by setting up the server and viewer.
- To set up the flatmap’s server, open a new terminal and navigate to the flatmap-server directory. The server depends on the knowledgebase.db output by mapmaker. Therefore, the directory that has been set up (via the --output argument in mapmaker) must be referenced.
In the flatmap-server directory, point to the knowledgebase.db file created by mapmaker.
$ export FLATMAP_ROOT=$HOME/\_Enter path to directory_
Start the server.
$ poetry run mapserver
- With the flatmap server running, the flatmap can be viewed by starting the viewer.
Open another terminal and navigate to the flatmap-viewer directory. The viewer can be started with
$ npm start
The flatmap can now be viewed on the local server output by the flatmap viewer. This is typically localhost:3000.
Congratulations you made a map!
Installation for various environments
Installing developer mapmaker via Ubuntu (WSL)
Mapmaker can be installed on machines operating WindowsOS via the activation of the Windows Subsystem for Linux (WSL). To set up WSL, open a Windows command prompt as administrator and run the following command:
$ wsl --install
Once installed, reboot and install Ubuntu 22.04 via the Microsoft Store. Open Ubuntu 22.04 and enter a desired username and password (Note that the password will be needed upon sudo installations). Upon reboot, an Ubuntu terminal may open to complete the installation. Ubuntu 22.04 must be the environment mapmaker is run in, as Python 3.10 is required. The installation should now complete successfully, and a chevron prompt will appear.
Note that the flatmap resources require Python 3.10. The version of python in use can be found via
$ python3 -V
The flatmap resources can now be installed. Below are the commands and supplemental information for the respective instructions found in the Installing mapmaker resources section.
Note:
Ubuntu may need to be updated via
$ sudo apt-get update && sudo apt-get upgrade
$ sudo apt install make
$ sudo apt install g++
$ sudo apt install sqlite libsqlite3-dev
$ sudo apt install libz-dev
$ sudo apt install coinor-cbc
$ git clone <https://github.com/mapbox/tippecanoe.git>
$ cd tippecanoe
$ make -j
$ sudo make install
Note:
The repositories must be cloned in the root directory.
$ git clone https://github.com/AnatomicMaps/flatmap-maker
$ git clone https://github.com/AnatomicMaps/flatmap-server
$ git clone https://github.com/AnatomicMaps/flatmap-viewer
$ curl -sSL https://install.python-poetry.org | python3 -
$ PATH=$PATH:$HOME/.local/bin
$ cd flatmap-maker
$ poetry install
$ cd ../flatmap-server
$ sudo apt-get install libgl1-mesa-glx
$ poetry install
$ poetry run mapserver
$ cd ../flatmap-viewer
$ npm install
The output of npm install may mention several vulnerabilities. Following the recommended instructions, these can be resolved via:
$ npm audit fix
The viewer can be started with
$ npm start
Note:
npm start may fail to start the viewer. This may be due to the updated version of Nodejs. This can be resolved by utilizing the Node Version Manager (nvm). Following the GitHub installation instructions, install nvm and run the command to install v12.22.9 of Nodejs. The terminal may need to be restarted.
$ nvm install 12.22.9
This version of node can be run via
$ nvm run node --12.22.9
Using Mapmaker - Windows
$ cd flatmap-maker
$ poetry shell
$ export SCICRUNCH_API_KEY= *Enter a valid SciCrunch Key*
$ python runmaker.py --source ../flatmap-sources/rat-flatmap/manifest.json --output ../flatmap-knowledgebase --verbose --background-tiles --authoring --debug
In a separate terminal,
$ cd flatmap-server
$ export FLATMAP_ROOT=$HOME/../flatmap-knowledgebase
$ poetry run mapserver
In another separate terminal,
$ cd flatmap-viewer
$ npm start
The generated flatmap can now be viewed on the local host server, on port 3000.
Installing developer mapmaker on MacOS
Following the instructions of the official installer, install and update Homebrew.
% /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
% brew update
Note:
The Homebrew will need to be added to PATH, as recommended by the official installer. This can be done by running the commands provided in the output, as shown in Figure 16.
Install Python3.10, if not already installed.
% brew install [email protected]
% brew install tippecanoe
% curl -sSL <https://install.python-poetry.org> | python3 -
The poetry path needs to be added to the shell configuration. If using bash command:
% echo "export PATH=$PATH:$HOME/.local/bin" >> ~/.bashrc
If using zsh command:
% echo "export PATH=$PATH:$HOME/.local/bin" >> ~/.zshrc
% git clone https://github.com/AnatomicMaps/flatmap-maker
% git clone https://github.com/AnatomicMaps/flatmap-serve
% git clone https://github.com/AnatomicMaps/flatmap-viewer
Switch to the latest release of mapmaker. Note that the command below refers to the latest release in October 2022.
% cd flatmap-maker
% git checkout v1.5.1
The next step is dependent on the processing chip of the MacOS machine. Note that this then affects how mapmaker is run (See the Using Mapmaker - MacOS section). In the root directory:
For Intel based machines, install coin-or cbc.
% brew tap coin-or-tools/coinor
% brew install coin-or-tools/coinor/cbc
For ARM based machines, documentation can be found in mapmaker. To install conda, follow the official instructions.
% curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh"
% sh Miniforge3-MacOSX-arm64.sh
Confirm yes when prompted. In a new shell:
% cd flatmap-maker
% conda env create -f conda/M1Mac.yaml
Install Poetry in the flatmap-maker directory.
% poetry install
% cd ../flatmap-server
% poetry install
The mapserver can be started by running
% poetry run mapserver
And shut down by
% Command + .
% cd ../flatmap-viewer
% npm install
% npm audit fix
Note:
npm start may fail to start the viewer. This may be due to the updated version of Nodejs. This can be resolved by utilizing the Node Version Manager (nvm). Following the GitHub installation instructions, install nvm and run the command to install v12.22.9 of Nodejs. The terminal may need to be restarted.
% nvm install 12.22.9
This version of node can be run via
% nvm run node --12.22.9
The flatmap viewer can then be started with
% npm start
And shut down with
% Command + .
Using Mapmaker - MacOS
As aforementioned, how mapmaker is run is dependent on the processing chip of the MacOS machine.
Activate the virtual environment.
For intel based machines:
% cd flatmap-maker
% poetry shell
% export SCICRUNCH_API_KEY= *Enter a valid SCICRUNCH Key*
For ARM based machines, with Conda:
% cd flatmap-maker
% conda activate mapmaker
% export SCICRUNCH_API_KEY= *Enter a valid SCICRUNCH Key*
Mapmaker can now be run. Note that the command below is specific to this example case. The paths for the source and output directories are dependent on the set up of the flatmap resources.
% python runmaker.py --source ../rat-flatmap/manifest.json --output ../flatmaps --verbose --background-tiles --authoring --debug
In a separate terminal,
% cd flatmap-server
% export FLATMAP_ROOT=$HOME/flatmaps
% poetry run mapserver
In another separate terminal,
% cd flatmap-viewer
% npm start
The generated flatmap can now be viewed on the local host server, on port 3000.
Flatmap sources
The flatmap source files must follow a particular structure in order to successfully generate maps with no issues. A clone repository of the rat flatmap on the SPARC Portal has been set up and can be found here.
SVG File
The SVG file of the flatmap sources represents the base drawing of the flatmap. As can be seen in the rat_flatmap.svg file, each shape has a unique identifier and a closed shape. The nomenclature of the features in the SVG is important as it must be consistent with the JSON files. Below is a table of identifiers for types of features in the SVG.
Feature | Identifier |
---|---|
Closed shape | .id(FEATURE_ID) |
Centreline | .centreline id(CENTRELINE_ID) |
Table 3: The respective identifiers for features in the SVG file. These IDs must be strictly followed as the JSON files utilize them.
JSON Files
The JSON files of the flatmap sources contain information on the visual aspects of the SVG file. Here, knowledge on what the shapes in the SVG represent and the manner of their connections are laid out in the form of annotations. The identifiers used must match those in the SVG, and the layout of the files are very specific. For more information see the docs section of mapmaker.
Anatomical_map.json
The anatomical_map.json contains classes of features and their relative ontological ID - typically UBERON or ILX, as per the SciCrunch database. The layout is typically as follows:
{
“CLASS_ID”: {
“term”: “ONT_TERM”,
“name”: “FEATURE_NAME”
}
}
Properties.json
The properties.json file contains information on what features belong to which classes, as well as the connectivity knowledge between features. The structure can be split into two sections - the features definitions and the networks definitions.
Features in properties.json
The features section layout is typically as follows:
{
“features”: {
“FEATURE_ID”: {
“class”: “CLASS_ID”
}
}
The feature_ID must be consistent with the ID used in the SVG file, for the relative feature(s). The class_ID must be consistent with the relative class that has been set up in anatomical_map.json.
Networks in properties.json
The networks section layout is typically as follows:
{
“networks”: [
{
“centrelines”: [
{
“id”: “CENTRELINE_ID”,
“models”: “NERVE_ONT_TERM”
“connects”: [
“FEATURE_ID1”,
“FEATURE_ID2”
],
“contained-in”: [
“FEATURE_ID3”
]
}
}
}
The fields in the networks section are explained in Table 4. As with the anatomical_map.json, the identifiers used must be consistent with those in the SVG file.
Required fields | |
id | A unique identifier specific to what the centreline represents. |
connects | A list of closed shapes that the respective centreline connects. A minimum of two features is required as centrelines must have a start and end point. |
Optional fields | |
models | A unique identifier for the anatomical structure that the centrelines represents. Must only be used for centrelines not divided into segments. |
contained-in | A list of close shapes that the respective centrelines passes through. These must be unique and specific to the respective centreline (i.e. if more than one centreline passes through a shape it is of no use to list the shape in the contained-in field). |
Table 4: The required and optional fields for the structure of centrelines in the properties.json file.
Manifest.json
The manifest.json ties together the other JSON files and the SVG file. The structure of the manifest is typically as follows:
{
“id”:”FLATMAP_NAME”,
“models”: “FLATMAP_ONT_ID”,
"description”: “description.json”,
“properties”: “properties.json”,
“sckan-version”: “production”,
“sources”: [
{
“id”: “FLATMAP_ID”,
“href”: “FLATMAP_SVG”,
“kind”: “base”
}
],
“anatomicalMap”: “anatomical_map.json”,
“connectivityTerms”: “connectivity_terms.json”,
“neuronConnectivity”: [
“https://apinatomy.org/uris/models/ard-arm-cardiac”,
“https://apinatomy.org/uris/models/bolser-lewis”,
“https://apinatomy.org/uris/models/bronchomotor”,
“https://apinatomy.org/uris/models/keast-bladder”,
“https://apinatomy.org/uris/models/pancreas”,
“https://apinatomy.org/uris/models/sawg-distal-colon”,
“https://apinatomy.org/uris/models/sawg-stomach”,
“https://apinatomy.org/uris/models/spleen”
]
}
Given these files are set up in the correct manner, the desired flatmap should be able to generate with no issues.
Examples
As can be seen in Figure 24, from the anatomical_map.json file of the rat_flatmap files, the class urinary_2 represents the urinary bladder of the body.
The features in the SVG that represent the urinary_2 class are defined in the properties.json.
Note that the use of “-1” in the feature ID allows multiple features to belong to the same class (i.e. in Figure 26, urinary_7-1 and urinary_7-2 both belong to the urinary_7 class).
Within the networks section of the properties.json, urinary_13-1 is referenced as the endpoint of the centreline n_58.
Updated 5 months ago