The B2SHARE Docker image can be used as is and is available on Docker Hub as eudatb2share/b2share. However, a full B2SHARE installation requires many different services (e.g. elasticsearch, postgresql, redis, rabbitmq), thus we provide a standard deployment procedure that creates a demonstration environment. The next steps will describe how to use this deployment.
First clone the dockerize repository:
$ git clone https://github.com/EUDAT-B2SHARE/dockerize.gitB2SHARE requires B2ACCESS for user management. For this purpose you must create
a new B2ACCESS OAuth client, providing as the 'return URL' the address of the
local B2SHARE server and the authorization path:
https://$FQDN/api/oauth/authorized/b2access/ (where $FQDN must be replaced
with the domain of the B2SHARE server).
After successfully registering the B2ACCESS account please set the following environment variables with the username and password provided for the B2ACCESS account:
Configuration before version v2.2.5:
$ export B2ACCESS_CONSUMER_KEY=... # the username used for registration
$ export B2ACCESS_SECRET_KEY=... # the password used for registrationFor demonstration purposes it's recommended to use the staging B2ACCESS
server. In this case make sure to define and export the
USE_STAGING_B2ACCESS variable:
$ export USE_STAGING_B2ACCESS=1Configuration from v2.3.0:
With B2SHARE configuration loader introduced in v2.3.0, a B2SHARE_ prefix needs
to be added in front of variables. A new variable for enabling B2ACCESS was introduced:
$ export B2SHARE_B2ACCESS_CONSUMER_KEY=... # the username used for registration
$ export B2SHARE_B2ACCESS_SECRET_KEY=... # the password used for registration
$ export B2SHARE_USE_STAGING_B2ACCESS=... # Configure B2ACCESS
$ export B2SHARE_CONFIG_ENABLE_B2ACCESS=1 # Enable B2ACCESS featureThe B2SHARE_SECRET_KEY environment variable (a secret string used for encrypting user sessions) must also be set as an environment variable. It is recommended to use a randomly generated string for this purpose. E.g.:
$ export B2SHARE_SECRET_KEY=mfTpGpQBwTTOmWCoZrYSuiJ1UzXq1d2LB2SHARE needs connection to a list of external services: redis, elasticsearch,
a database (postgresql or mysql), and rabbitmq. To simplify the creation of
this necessary execution environment, the Docker application engine is used.
The default Docker configuration for B2SHARE will try to mount a folder from
the host machine. This host directory is designated by the environment
variable $B2SHARE_DATADIR which should be an absolute path. This folder
is used to store all the data uploaded into B2SHARE and is therefore very
important.
B2SHARE has a set of configuration parameters, which should be inspected and
changed when necessary by the site administrator. On the first run of the
service, a configuration file will be created in the B2SHARE data location
(which also by default is mounted to the host folder
${B2SHARE_DATADIR}/b2share-data/ as described above). The name of this
configuration file is b2share.cfg. Please note that after each update of
this file the service needs to be restarted
(with e.g.``docker-compose restart``) for the changes to take effect.
By default, the B2SHARE service will use a newly created postgresql database.
This database's data is persisted in ${B2SHARE_DATADIR}/postgresql-data/.
In case you want to use an external database, one can be configured
by editing the b2share.cfg file and defining the
SQLALCHEMY_DATABASE_URI variable. For example, for a postresql database,
use SQLALCHEMY_DATABASE_URI =
"postgresql://db_username:db_password@db_host/db_name")
Note that you also need to remove the B2SHARE_SQLALCHEMY_DATABASE_URI
variable in the docker-compose.yml file.
B2SHARE uses nginx as the front-end HTTP server, responsible for establishing
secure connections with clients over https. By default an autogenerated
security certificate is used, which triggers security warnings in the client
browsers. For a production server it is recommended to use a
valid certificate. To use it with nginx, place the certificate files in an
empty folder on the host machine (e.g. /b2share-etc/ssl) and mount the
folder in the nginx container by updating the volumes declaration in the
docker-compose.yml file. Make sure that the following files are present in
the mounted folder:
ssl_certificate /b2share-etc/ssl/b2share.crt
ssl_certificate_key /b2share-etc/ssl/b2share.keyAfter all the configuration parameters have been set go into the recently
downloaded dockerize folder and run docker-compose, e.g.:
Before v2.2.5:
$ cd dockerize
## mandatory environment variables
$ export B2ACCESS_CONSUMER_KEY=... # the username used for registration
$ export B2ACCESS_SECRET_KEY=... # the password used for registration
$ export B2SHARE_SECRET_KEY=... # a secret string used for encrypting user sessions
$ export B2SHARE_JSONSCHEMAS_HOST='<FQDN>' # replace <FQDN> with the domain of the B2SHARE server
$ export B2SHARE_POSTGRESQL_DBNAME=... # name of the postgresql database used by b2share
$ export B2SHARE_POSTGRESQL_PASSWORD=... # password used by b2share when accessing the postgresql database
$ export B2SHARE_POSTGRESQL_USER=... # username used by b2share when accessing the postgresql database (default=b2share)
$ export B2SHARE_RABBITMQ_USER=... # username used by b2share when accessing RabbitMQ service
$ export B2SHARE_RABBITMQ_PASS=... # password used by b2share when accessing RabbitMQ service
$ export B2SHARE_DATADIR='<PATH>' # path on the host which will be mounted and contain all b2share related data, including postgresql, elasticsearch, redis, rabbitmq, nginx and b2share itself.
## optional environment variables
$ export USE_STAGING_B2ACCESS=1 # to run with staging (testing) b2access (unity install)
$ export INIT_DB_AND_INDEX=1 # when run the first time, initialize the database and indices
$ export LOAD_DEMO_COMMUNITIES_AND_RECORDS=1 # when run the first time, load some communities and records for demonstration
## run it
$ docker-compose up -dFrom v2.3.0:
$ cd dockerize
## mandatory environment variables
$ export B2SHARE_B2ACCESS_CONSUMER_KEY=... # the username used for registration
$ export B2SHARE_B2ACCESS_SECRET_KEY=... # the password used for registration
$ export B2SHARE_SECRET_KEY=... # a secret string used for encrypting user sessions
$ export B2SHARE_JSONSCHEMAS_HOST='<FQDN>' # replace <FQDN> with the domain of the B2SHARE server
$ export B2SHARE_POSTGRESQL_DBNAME=... # name of the postgresql database used by b2share
$ export B2SHARE_POSTGRESQL_PASSWORD=... # password used by b2share when accessing the postgresql database
$ export B2SHARE_POSTGRESQL_USER=... # username used by b2share when accessing the postgresql database (default=b2share)
$ export B2SHARE_RABBITMQ_USER=... # username used by b2share when accessing RabbitMQ service
$ export B2SHARE_RABBITMQ_PASS=... # password used by b2share when accessing RabbitMQ service
$ export B2SHARE_DATADIR='<PATH>' # path on the host which will be mounted and contain all b2share related data, including postgresql, elasticsearch, redis, rabbitmq, nginx and b2share itself.
$ export B2SHARE_LOGGING_LEVEL=... # set logging level, must be int or allowed string
## optional environment variables
$ export B2SHARE_USE_STAGING_B2ACCESS=1 # to run with staging (testing) b2access (unity install)
$ export B2SHARE_INIT_DB_AND_INDEX=1 # when run the first time, initialize the database and indices
$ export B2SHARE_LOAD_DEMO_COMMUNITIES_AND_RECORDS=1 # when run the first time, load some communities and records for demonstration
$ export B2SHARE_CONFIG_ENABLE_B2ACCESS=1 # Enable B2ACCESS feature
$ export B2SHARE_CONFIG_USE_VAULT_AGENT=... # If a secret provider is used, such as Vault
## run it
$ docker-compose up -dAfter the docker image is built and running, B2SHARE will be available at https://<FQDN>. If B2SHARE does not become available, you can try inspecting the container logs to find the cause:
$ docker-compose logs -f b2shareThe services can be stopped just like any other docker-compose deployment.
$ docker-compose downThis command will stop and remove all containers. The data will still be
available in the directory pointed by $B2SHARE_DATADIR.
The services can be started as before. If the directory pointed by the
environment variable $B2SHARE_DATADIR already contains data from
a previous run, the data will be reused. See section 1.7 for removing
all previously persisted data if you want to start from scratch.
$ docker-compose up -dIn order to remove all persisted data one can just delete the directory pointed
by $B2SHARE_DATADIR. All containers must be stopped before removing the
directory.
Remember to also delete the containers.
The provided docker-compose file is not production ready. The
database has no backup mechanism running. The system administrator can
comment the "postgres" service and set the SQLALCHEMY_DATABASE_URI as
described in section 1.3. It is also possible to enhance the postgresql
container by building a custom one.
By default files uploaded by users are written in
${B2SHARE_DATADIR}/b2share-data/files. This location must be backuped or
written on a safe filesystem.
If the containers need to run on different servers it is recommended to use native docker volumes instead of mounted host directories.
The records metadata in your local instance of B2SHARE can be harvested by
various other repositories by using the included endpoint for the OAI-PMH
protocol. For the records to be visible in B2FIND, please create a ticket in
the B2FIND support queue (https://eudat.eu/support-
request?service=B2FIND), by describing your instance and specifying the
OAI-PMH url (http://<FQDN>/oai2d)
Before installing B2Share you will need the following software:
python3virtualenvandvirtualenvwrapper
$ # on OSX, with brew:
$ brew install python --framework --universal
$ pip install virtualenv virtualenvwrapperand a working installation of the following services:
Elasticsearch 2.4Redis 2.10.5RabbitMQ 3.6Celery 4.1PostgreSQL 9.6andpsycopg2
Another option, which requires a bit more RAM, is to use a docker-compose.yml file to manage the starting and stopping of the services, and just have a bare metal installation of B2Share for development.
Now to install B2Share, first create a new virtualenv with:
$ virtualenv -p python3 b2shareand clone the B2Share repository from github:
$ git clone https://github.com/eudat-b2share/b2share.gitNow we can install b2share for development with pip:
$ pip install -r requirements.txt
$ pip install -e .[all]and start all services.
B2Share requires some configuration as mentioned in section 1.1 of this document so make sure to follow the steps exporting the enviroment variables, and the additional ones required for local installations:
$ export B2SHARE_UI_PATH=<path-to-b2share-repository>/webui/app
$ export B2SHARE_SQLALCHEMY_DATABASE_URI=postgresql+psycopg2://$B2SHARE_POSTGRESQL_USER:$B2SHARE_POSTGRESQL_PASSWORD@localhost:5432/$B2SHARE_POSTGRESQL_DBNAMEFinally, create the DB:
$ b2share upgrade runcreate the Elasticsearch indices and load the templates:
$ b2share index init
$ b2share schemas initand start B2Share with:
$ b2share runIf you wish to add some demo records and communities, then install the b2share demo application:
$ cd <path-to-b2share-repository>/demo/
$ pip install .and load the demo data:
$ b2share demo load_data -vvPlease note that a custom B2ACCESS configuration is also needed, as described
above. The 'return URL' of the B2ACCESS configuration in this case can be set
to http://localhost:5000/api/oauth/authorized/b2access/
If working on the web UI, see also: https://github.com/EUDAT-B2SHARE/b2share/wiki/Developer's-corner.
FAQ:
- There is a timeout when trying to create a new record from the UI, or the server hangs when trying to create a draft from the REST API.
This could indicate that the JSONSCHEMAS_HOST is misconfigured. If you are running B2SHARE locally then it should be set to localhost:5000.
When trying to login the following error is shown:
ERROR OAuth Authorization Server got an invalid request.
If you are a user then you can be sure that the web application you was using previously is either misconfigured or buggy.
If you are an administrator or developer the details of the error follows:
The client '<your-b2access-consumer-key>' is unknown
This means that your B2ACCESS credentials are incorrect, or you are working with the staging server and you didn't set USE_STAGING_B2ACCESS=1.
- There is the following error when you run b2share upgrade run: sqlalchemy.exc.InvalidRequestError: Naming convention including %(constraint_name)s
If your alembic recipes are unchanged and you also didn't change any of your models.py, this could be due to not having set the B2SHARE_SQLALCHEMY_DATABASE_URI to use Postgresql, and you are using the default which is SQLite, which doesn't support DB migrations with alembic.