Installing CKAN from source#
This section describes how to install CKAN used by depositar
from source on an Ubuntu 20.04 server.
1. Install the required packages#
sudo apt install python3-dev postgresql libpq-dev python3-pip python3-venv git openjdk-8-jdk redis-server
2. Install CKAN into a Python virtual environment#
Create a Python virtual environment (virtualenv) to install CKAN into, and activate it:
sudo mkdir -p /usr/lib/ckan/default sudo chown `whoami` /usr/lib/ckan/default python3 -m venv /usr/lib/ckan/default . /usr/lib/ckan/default/bin/activate
Important
The final command above activates your virtualenv. The virtualenv has to remain active for the rest of the installation and deployment process, or commands will fail. You can tell when the virtualenv is active because its name appears in front of your shell prompt, something like this:
(default) $ _
For example, if you logout and login again, or if you close your terminal window and open it again, your virtualenv will no longer be activated. You can always reactivate the virtualenv with this command:
. /usr/lib/ckan/default/bin/activate
Install wheel:
Important
Please run all the commands below under the ckan directory:
cd /usr/lib/ckan/default/
pip install wheel
Install CKAN into your virtualenv:
pip install -e 'git+https://github.com/depositar/ckan.git#egg=ckan[requirements]'
Install customized extesion into your virtualenv:
pip install -e 'git+https://github.com/depositar/ckanext-data-depositario.git#egg=ckanext-data-depositario'
Install the Python modules that customized extesion requires into your virtualenv:
pip install -r /usr/lib/ckan/default/src/ckanext-data-depositario/requirements.txt
Install other required Python modules into your virtualenv:
pip install -r /usr/lib/ckan/default/src/ckanext-spatial/pip-requirements.txt pip install -r https://raw.githubusercontent.com/ckan/ckanext-xloader/0.11.0/requirements.txt pip install -r /usr/lib/ckan/default/src/ckanext-dcat/requirements.txt pip install -r /usr/lib/ckan/default/src/ckanext-harvest/pip-requirements.txt
3. Create the FireStore directory#
Note
When enabled, CKAN’s FileStore allows users to upload data files to CKAN resources. Please refer to User guide for details.
sudo mkdir -p /var/lib/ckan/default sudo chown `whoami` /var/lib/ckan/default sudo chmod u+rwx /var/lib/ckan/default
4. Setup a PostgreSQL database#
Create a database user:
sudo -u postgres createuser -S -D -R -P ckan_default
Create a new database:
sudo -u postgres createdb -O ckan_default ckan_default -E utf-8
Install the PostGIS:
sudo apt install postgresql-12-postgis-3 python3-dev libxml2-dev libxslt1-dev libgeos-c1v5 sudo -u postgres psql -d ckan_default -f /usr/share/postgresql/12/contrib/postgis-3.0/postgis.sql sudo -u postgres psql -d ckan_default -f /usr/share/postgresql/12/contrib/postgis-3.0/spatial_ref_sys.sql sudo -u postgres psql -d ckan_default -c 'ALTER VIEW geometry_columns OWNER TO ckan_default;' sudo -u postgres psql -d ckan_default -c 'ALTER TABLE spatial_ref_sys OWNER TO ckan_default;'
Create a new database user and a new database for DataStore:
Note
The CKAN DataStore extension provides an ad hoc database for storage of structured data from CKAN resources. Data can be pulled out of resource files and stored in the DataStore.
sudo -u postgres createuser -S -D -R -P -l datastore_default sudo -u postgres createdb -O ckan_default datastore_default -E utf-8
(For
depositar
administrator) Restore database backup:cat main_db.sql.gz | gunzip | sudo -u postgres psql ckan_default cat datastore_db.sql.gz | gunzip | sudo -u postgres psql datastore_default
5. Create a CKAN config file#
Create a directory to contain the site’s config files:
sudo mkdir -p /etc/ckan/default sudo chown -R `whoami` /etc/ckan/
Create a CKAN config file:
Important
(For
depositar
administrator) Please ignore the following step. c and useproduction.ini
the in theconfigs.tar.gz
.Important
The virtualenv has to remain active when running the ckan command. You can always reactivate the virtualenv with this command:
. /usr/lib/ckan/default/bin/activate
ckan generate config /etc/ckan/default/ckan.ini ckan config-tool /etc/ckan/default/ckan.ini -f /usr/lib/ckan/default/src/ckanext-data-depositario/config/custom_options.ini
Edit the ckan.ini file in a text editor, changing the following options:
Note
The settings below is the minimum requirements to run the CKAN.
## Database Settings ## This should refer to the database we created in 4. Setup a PostgreSQL database above ## Replace
pass
with theCKAN database
password that you created sqlalchemy.url = postgresql://ckan_default:pass@localhost/ckan_default ## Replacepass
with theCKAN database
password that you created ckan.datastore.write_url = postgresql://ckan_default:pass@localhost/datastore_default ## Replacepass
with theDataStore database
password that you created ckan.datastore.read_url = postgresql://datastore_default:pass@localhost/datastore_default ## Add the following lines above Logging configuration ## Search Settings ckan.search.solr_allowed_query_parsers = field ## Schema Settings scheming.presets = ckanext.scheming:presets.json ckanext.data_depositario:presets.json ckanext.wikidatakeyword:presets.json scheming.dataset_schemas = ckanext.data_depositario.schemas:dataset.yaml ## Spatial Settings ckanext.spatial.search_backend = solr-spatial-field ## DCAT Settings ckanext.dcat.rdf.profiles = dcat ckanext.dcat.translate_keys = False ckanext.dcat.enable_content_negotiation = True ## ckanext-data-depositario Settings ## GMAP_AKI_KEY is the API key for Google Maps ckanext.data_depositario.gmap.api_key = GMAP_AKI_KEY ## GA_ID is the id for Google Analytics ckanext.data_depositario.googleanalytics.id = GA_ID
6. Setup Solr (with Chinese and spatial search support)#
Note
This section is adapted from How To Install Solr 5.2.1 on Ubuntu 14.04 by DigitalOcean™ Inc. licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International.
Download and extract the service installation file:
cd ~ wget http://archive.apache.org/dist/lucene/solr/8.11.2/solr-8.11.2.tgz tar xzf solr-8.11.2.tgz solr-8.11.2/bin/install_solr_service.sh --strip-components=2
Install Solr as a service using the script:
sudo bash ./install_solr_service.sh solr-8.11.2.tgz
Create the Solr core for CKAN:
sudo -u solr /opt/solr/bin/solr create -c ckan sudo ln -sf /usr/lib/ckan/default/src/ckanext-data-depositario/solr/schema.xml /var/solr/data/ckan/conf/managed-schema
Download Chinese tokenizer
ik-analyzer
and copy it to the Solr directory:wget https://repo1.maven.org/maven2/com/github/magese/ik-analyzer/8.5.0/ik-analyzer-8.5.0.jar sudo cp ik-analyzer-8.5.0.jar /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/. sudo mkdir /opt/solr/server/solr-webapp/webapp/WEB-INF/classes sudo ln -s /usr/lib/ckan/default/src/ckanext-data-depositario/solr/IKAnalyzer.cfg.xml /opt/solr/server/solr-webapp/webapp/WEB-INF/classes/. sudo ln -s /usr/lib/ckan/default/src/ckanext-data-depositario/solr/dic/words.dic /opt/solr/server/solr-webapp/webapp/WEB-INF/classes/words.dic
Download geometry library JTS Topology Suite 1.18 (or above) and copy it to the Solr directory:
wget https://repo1.maven.org/maven2/org/locationtech/jts/jts-core/1.19.0/jts-core-1.19.0.jar sudo cp jts-core-1.19.0.jar /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/.
Restart Solr:
sudo service solr restart
Open http://127.0.0.1:8983/solr/#/ckan in a web browser, and you should see the Solr front page.
7. Link to who.ini
#
ln -s /usr/lib/ckan/default/src/ckan/who.ini /etc/ckan/default/who.ini
8. Create database tables#
Important
(For depositar
administrator) Please ignore this step.
Set up the DataStore:
ckan -c /etc/ckan/default/ckan.ini db init
You should see Initialising DB: SUCCESS.
Then you can use this connection to set up the DataStore:
ckan -c /etc/ckan/default/ckan.ini datastore set-permissions | sudo -u postgres psql --set ON_ERROR_STOP=1
Set up the ARK database:
ckan -c /etc/ckan/default/ckan.ini ark initdb
You should see ARK table created.
9. Creating a sysadmin user#
Important
(For depositar
administrator) Please ignore this step.
Set password for the default CKAN sysadmin user from the command line.
ckan -c /etc/ckan/default/ckan.ini user setpass default
10. Serve CKAN under a development server#
Run the XLoader:
Note
This XLoader is a service that automatically uploads data to the DataStore from suitable files (like CSV or Excel files), whether uploaded to CKAN’s FileStore or externally linked.
The CKAN DataStore extension provides an ad hoc database for storage of structured data from CKAN resources. Data can be pulled out of resource files and stored in the DataStore.
ckan -c /etc/ckan/default/ckan.ini jobs worker
Open another terminal and use the Paste development server to serve CKAN from the command-line:
. /usr/lib/ckan/default/bin/activate ckan -c /etc/ckan/default/development.ini
Open http://127.0.0.1:5000/ in a web browser, and you should see the CKAN front page.
Now that you’ve installed CKAN.