Getting Started
Zocalo requires both ActiveMQ and Graylog to be setup and running. The easiest way of setting these up is via docker.
Active MQ
Pull and run the following image https://hub.docker.com/r/rmohr/activemq Follow the steps on docker hub for extracting the config and data into local mounts
Configure DLQ locations, see https://activemq.apache.org/message-redelivery-and-dlq-handling for more info.
In conf/activemq.xml under policyEntries add:
<policyEntry queue=">">
<deadLetterStrategy>
<individualDeadLetterStrategy queuePrefix="DLQ." useQueueForQueueMessages="true"/>
</deadLetterStrategy>
</policyEntry>
Make sure to enable scheduling, in conf/activemq.xml in the broker tag add the following property:
schedulerSupport="true"
Its also a good idea to enable removal of unused queues, see https://activemq.apache.org/delete-inactive-destinations
In conf/activemq.xml in the broker tag add the following property:
schedulePeriodForDestinationPurge="10000"
Then in the policyEntry tag for queue=”>” add the following properties:
gcInactiveDestinations="true" inactiveTimoutBeforeGC="120000"
Which will purge unused queues on a 120s basis.
Then start ActiveMQ:
docker run --name activemq -p 61613:61613 -p 8161:8161 \
-v "$(pwd)/conf:/opt/activemq/conf" \
-v "$(pwd)/data:/opt/activemq/data" \
rmohr/activemq
The container exposes the following ports:
Port |
Description |
---|---|
61613 |
Stomp transport |
8161 |
Web Console / Jolokia REST API |
A preconfigured docker image with these options applied is available here https://hub.docker.com/r/esrfbcu/zocalo-activemq
Graylog
This can be started easily with a docker-compose.yml. See https://docs.graylog.org/en/3.3/pages/installation/docker.html for full details.
version: '3'
services:
# MongoDB: https://hub.docker.com/_/mongo/
mongo:
image: mongo:4.2
networks:
- graylog
# Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/6.x/docker.html
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.0
environment:
- http.host=0.0.0.0
- transport.host=localhost
- network.host=0.0.0.0
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
deploy:
resources:
limits:
memory: 1g
networks:
- graylog
# Graylog: https://hub.docker.com/r/graylog/graylog/
graylog:
image: graylog/graylog:4.0
environment:
- GRAYLOG_PASSWORD_SECRET=mysecret
# Password: admin
- GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
- GRAYLOG_HTTP_EXTERNAL_URI=http://localhost:9000/
networks:
- graylog
restart: always
depends_on:
- mongo
- elasticsearch
ports:
# Graylog web interface and REST API
- 9000:9000
# Syslog TCP
- 1514:1514
# Syslog UDP
- 1514:1514/udp
# GELF TCP
- 12201:12201
# GELF UDP
- 12201:12201/udp
networks:
graylog:
driver: bridge
Then start with:
docker-compose up
Graylog admin console should be available on http://localhost:9000 Port 12201 is available for python GELF logging. Configure an input in the graylog web console to enable receiving messages.
Zocalo
For developing create a new conda / virtual environment, clone zocalo, and install:
conda create -n zocalo
conda activate zocalo
git clone https://github.com/DiamondLightSource/python-zocalo
cd python-zocalo
pip install -e .
For production, install with pip:
pip install zocalo
Configure
Copy contrib/site-configuration.yml. At minimum graylog and activemq must be configured. Environments should be defined for live and test. Paths to recipes and drop files must also be specified. Messages are written to drop files if ActiveMQ is unavailable.
The config file to use is specified from the environment variable ZOCALO_CONFIG.
Sample recipes can be used:
storage:
plugin: storage
zocalo.recipe_directory: .../python-zocalo/examples/recipes
JMX
To make use of zocalo.queue_monitor and zocalo.status_monitor JMX needs to be configured. The JMX configuration points to the Jolokia REST API. When starting ActiveMQ the logs will tells you where the REST API is running
INFO | ActiveMQ Jolokia REST API available at http://0.0.0.0:8161/api/jolokia/
So configuration should be
port: 8161
host: localhost
base_url: api/jolokia
Username and password are the same as the web console and defined in users.properties
Starting Up
-e test will make use of the test environment
Start the dispatcher
conda activate zocalo
zocalo.service -s Dispatcher (-e test)
Start the process runner
zocalo.service -s Runner (-e test)
Run the test recipe:
zocalo.go -r example -s workingdir="$(pwd)" 1234 (-e test)
Dead Letter Queue (DLQ)
The dead letter queue is where rejected messages end up. One dlq is available per topic to easily identify where messages are being rejected. For details on dlq see https://activemq.apache.org/message-redelivery-and-dlq-handling
Messages can be purged using:
zocalo.dlq_purge --output-directory=/path/to/dlq (-e test)
And re-injected with:
zocalo.dlq_reinject dlq_file (-e test)