Installing CKAN from source¶
This section describes how to install CKAN used by depositar
from source on an Ubuntu 16.04 server.
1. Install the required packages¶
sudo apt-get install build-essential libxslt1-dev libxml2-dev python-dev postgresql libpq-dev python-pip python-virtualenv git-core openjdk-8-jdk redis-server
2. Install CKAN into a Python virtual environment¶
Create a Python virtual environment (virtualenv) to install CKAN into, and activate it:
sudo mkdir -p /usr/lib/ckan/default sudo chown `whoami` /usr/lib/ckan/default virtualenv --no-site-packages /usr/lib/ckan/default . /usr/lib/ckan/default/bin/activate
Important
The final command above activates your virtualenv. The virtualenv has to remain active for the rest of the installation and deployment process, or commands will fail. You can tell when the virtualenv is active because its name appears in front of your shell prompt, something like this:
(default) $ _
For example, if you logout and login again, or if you close your terminal window and open it again, your virtualenv will no longer be activated. You can always reactivate the virtualenv with this command:
. /usr/lib/ckan/default/bin/activate
Install the recommended setuptools version:
Important
Please run all the commands below under the ckan directory:
cd /usr/lib/ckan/default/
pip install setuptools==36.1
Install CKAN into your virtualenv:
pip install -e 'git+git://github.com/depositar-io/ckan.git#egg=ckan'
Install customized extesion into your virtualenv:
pip install -e 'git+https://github.com/depositar-io/ckanext-data-depositario.git#egg=ckanext-data-depositario'
Install the Python modules that CKAN requires into your virtualenv:
pip install -r /usr/lib/ckan/default/src/ckan/requirements.txt
Install the Python modules that customized extesion requires into your virtualenv:
pip install -r /usr/lib/ckan/default/src/ckanext-data-depositario/requirements.txt
Install other required Python modules into your virtualenv:
pip install -r /usr/lib/ckan/default/src/ckanext-spatial/pip-requirements-py2.txt pip install -r https://raw.githubusercontent.com/ckan/ckanext-xloader/master/requirements.txt pip install -r /usr/lib/ckan/default/src/ckanext-dcat/requirements.txt
3. Create the FireStore directory¶
Note
When enabled, CKAN’s FileStore allows users to upload data files to CKAN resources. Please refer to User guide for details.
sudo mkdir -p /var/lib/ckan/default sudo chown `whoami` /var/lib/ckan/default sudo chmod u+rwx /var/lib/ckan/default
4. Setup a PostgreSQL database¶
Create a database user:
sudo -u postgres createuser -S -D -R -P ckan_default
Create a new database:
sudo -u postgres createdb -O ckan_default ckan_default -E utf-8
Install the PostGIS:
sudo apt-get install postgresql-9.5-postgis-2.2 python-dev libxml2-dev libxslt1-dev libgeos-c1v5 sudo -u postgres psql -d ckan_default -f /usr/share/postgresql/9.5/contrib/postgis-2.2/postgis.sql sudo -u postgres psql -d ckan_default -f /usr/share/postgresql/9.5/contrib/postgis-2.2/spatial_ref_sys.sql sudo -u postgres psql -d ckan_default -c 'ALTER VIEW geometry_columns OWNER TO ckan_default;' sudo -u postgres psql -d ckan_default -c 'ALTER TABLE spatial_ref_sys OWNER TO ckan_default;'
Create a new database user and a new database for DataStore:
Note
The CKAN DataStore extension provides an ad hoc database for storage of structured data from CKAN resources. Data can be pulled out of resource files and stored in the DataStore.
sudo -u postgres createuser -S -D -R -P -l datastore_default sudo -u postgres createdb -O ckan_default datastore_default -E utf-8
(For
depositar
administrator) Restore database backup:gunzip -c main_db.sql.gz | sudo -u postgres psql ckan_default gunzip -c datastore_db.sql.gz | sudo -u postgres psql datastore_default
5. Create a CKAN config file¶
Create a directory to contain the site’s config files:
sudo mkdir -p /etc/ckan/default sudo chown -R `whoami` /etc/ckan/
Create the CKAN config file via paster:
Important
(For
depositar
administrator) Please ignore the following step. c and useproduction.ini
the in theconfigs.tar.gz
.Important
The virtualenv has to remain active when running the paster command. You can always reactivate the virtualenv with this command:
. /usr/lib/ckan/default/bin/activate
paster make-config ckan /etc/ckan/default/development.ini
Edit the development.ini file in a text editor, changing the following options:
Note
- The settings below is the minimum requirements to run the CKAN.
## Database Settings ## This should refer to the database we created in 4. Setup a PostgreSQL database above ## Replace
pass
with theCKAN database
password that you created sqlalchemy.url = postgresql://ckan_default:pass@localhost/ckan_default ## Replacepass
with theCKAN database
password that you created ckan.datastore.write_url = postgresql://ckan_default:pass@localhost/datastore_default ## Replacepass
with theDataStore database
password that you created ckan.datastore.read_url = postgresql://datastore_default:pass@localhost/datastore_default ## Site Settings ckan.site_url = http://127.0.0.1:5000 ## Plugins Settings ckan.plugins = dat data_depositario depositar_iso639 depositar_theme citation wikidatakeyword dcat_json_interface structured_data stats datastore xloader resource_proxy recline_view text_view image_view webpage_view recline_grid_view recline_map_view pdf_view spatial_metadata spatial_query geo_view geojson_view wmts_view shp_view scheming_datasets ## Front-End Settings licenses_group_url = file:///usr/lib/ckan/default/src/ckanext-data-depositario/ckanext/data_depositario/public/license_list.json ## Storage Settings ckan.storage_path = /var/lib/ckan/default ## Schema Settings ## Add these settings scheming.presets = ckanext.scheming:presets.json ckanext.data_depositario:presets.json ckanext.wikidatakeyword:presets.json scheming.dataset_schemas = ckanext.data_depositario.schemas:dataset.yaml ## Spatial Settings ## Add these settings ckanext.spatial.search_backend = solr-spatial-field ## DCAT Settings ckanext.dcat.rdf.profiles = dcat ckanext.dcat.translate_keys = False ckanext.dcat.enable_content_negotiation = True ## ckanext-data-depositario Settings ## Add these settings ## GMAP_AKI_KEY is the API key for Google Maps ckanext.data_depositario.gmap.api_key = GMAP_AKI_KEY ## GA_ID is the id for Google Analytics ckanext.data_depositario.googleanalytics.id = GA_ID
6. Setup Solr (with Chinese and spatial search support)¶
Note
This section is adapted from How To Install Solr 5.2.1 on Ubuntu 14.04 by DigitalOcean™ Inc. licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International.
Download and extract the service installation file:
cd ~ wget http://archive.apache.org/dist/lucene/solr/5.5.5/solr-5.5.5.tgz tar xzf solr-5.5.5.tgz solr-5.5.5/bin/install_solr_service.sh --strip-components=2
Install Solr as a service using the script:
sudo bash ./install_solr_service.sh solr-5.5.5.tgz
Create the Solr configset for CKAN:
sudo -u solr mkdir -p /var/solr/data/configsets/ckan/conf sudo ln -s /usr/lib/ckan/default/src/ckanext-data-depositario/solr/schema.xml /var/solr/data/configsets/ckan/conf/schema.xml sudo -u solr cp /opt/solr/server/solr/configsets/basic_configs/conf/solrconfig.xml /var/solr/data/configsets/ckan/conf/. sudo -u solr touch /var/solr/data/configsets/ckan/conf/protwords.txt sudo -u solr touch /var/solr/data/configsets/ckan/conf/synonyms.txt
Download Chinese tokenizer
mmseg4j
and copy it to the Solr directory:wget -O mmseg4j-core-1.10.0.jar https://search.maven.org/remotecontent?filepath=com/chenlb/mmseg4j/mmseg4j-core/1.10.0/mmseg4j-core-1.10.0.jar wget -O mmseg4j-solr-2.4.0.jar https://search.maven.org/remotecontent?filepath=com/chenlb/mmseg4j/mmseg4j-solr/2.4.0/mmseg4j-solr-2.4.0.jar sudo cp mmseg4j-*.jar /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/.
Download geometry library JTS Topology Suite 1.13 (or above) and copy it to the Solr directory:
wget -O jts-1.13.jar https://search.maven.org/remotecontent?filepath=com/vividsolutions/jts/1.13/jts-1.13.jar sudo cp jts-1.13.jar /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/.
Replace all lines in /var/solr/data/configsets/ckan/conf/solrconfig.xml from line 99 to line 102 about
<schemaFactory class="ManagedIndexSchemaFactory">
with<schemaFactory class="ClassicIndexSchemaFactory"/>
.Restart Solr:
sudo service solr restart
Create a new Solr core called
ckan
by entering the following link in a web browser:http://127.0.0.1:8983/solr/admin/cores?action=CREATE&name=ckan&configSet=ckan
Open http://127.0.0.1:8983/solr/#/ckan in a web browser, and you should see the Solr front page.
Modify /etc/ckan/default/development.ini with Solr url:
solr_url = http://127.0.0.1:8983/solr/ckan
7. Create database tables¶
Important
(For depositar
administrator) Please ignore this step.
Create the database tables via paster:
paster --plugin=ckan db init -c /etc/ckan/default/development.ini
You should see Initialising DB: SUCCESS.
Then you can use this connection to set up the DataStore:
paster --plugin=ckan datastore set-permissions -c /etc/ckan/default/development.ini | sudo -u postgres psql --set ON_ERROR_STOP=1 wget -O- https://github.com/ckan/ckanext-xloader/raw/master/full_text_function.sql | sudo -u postgres psql datastore_default
8. Link to who.ini
¶
ln -s /usr/lib/ckan/default/src/ckan/who.ini /etc/ckan/default/who.ini
9. Creating a sysadmin user¶
Important
(For depositar
administrator) Please ignore this step.
You have to create your first CKAN sysadmin user from the command line. For example, to create a user called admin and make him a sysadmin:
paster --plugin=ckan sysadmin add admin -c /etc/ckan/default/development.ini
paster --plugin=pylons shell /etc/ckan/default/development.ini
Execute the following commands in the interactive shell:
model.User.get('admin').state = 'active'
model.Session.commit()
Then press Ctrl+D to exit the interactive shell.
10. Serve CKAN under a development server¶
Run the XLoader:
Note
This XLoader is a service that automatically uploads data to the DataStore from suitable files (like CSV or Excel files), whether uploaded to CKAN’s FileStore or externally linked.
The CKAN DataStore extension provides an ad hoc database for storage of structured data from CKAN resources. Data can be pulled out of resource files and stored in the DataStore.
paster --plugin=ckan jobs -c /etc/ckan/default/development.ini worker
Open another terminal and use the Paste development server to serve CKAN from the command-line:
. /usr/lib/ckan/default/bin/activate paster serve /etc/ckan/default/development.ini
Open http://127.0.0.1:5000/ in a web browser, and you should see the CKAN front page.
Now that you’ve installed CKAN.