Developing Services (Old)
Services are packaged as docker containers, using docker-compose (previously fig) to define relationships between the containers that make up a service.
Mention your idea for a service on the email list and get at least one person to volunteer to help admin the service once its running, and ideally help you develop it.
Then update the Services Roadmap to show that what you're working on.
Before developing a service, you'll need to install the following:
Getting the code
git clone ssh://firstname.lastname@example.org/darkpeak-services.git
Starting a service
Inside the service's directory in the darkpeak-services repository, run the following:
This will fetch the underlying images and build the containers required for the service. It will then start the containers and bind the appropriate port numbers on your host machine (look in docker-compose.yml for port mappings).
Anatomy of a service
myservice/ docker-compose.yml mywebserver/ Dockerfile nginx.conf public/ index.html style.css mydatabase/ Dockerfile scripts/ updates_available.sh backup.sh restore.sh
The above gives you some idea of the parts that might make up a Dark Peak service. The
docker-compose.yml file describes how to start the
mydatabase containers and link them together. It also describes which ports to expose on the host system. The scripts directory contains shell scripts to manage the service. The scripts talk to a docker daemon in order to handle common tasks such as checking for package updates and performing backups.
Try to follow the conventions used in other Dockerfiles where possible. Having a more homogeneous system makes administration and maintenance tasks easier.
- Use of the debian:wheezy base image is preferred
- Rebuilding a container without using the cache should produce a fully up-to-date service with no out of date packages. This can usually be achieved by adding
RUN apt-get update && apt-get upgrade -yto your Dockerfile
- Do not mount host volumes in the
docker-compose.ymlconfig, descibe volumes inside the Dockerfiles using the
VOLUMEkeyword and mount them in other containers using
- You should be able to deploy the service knowing only the URI for the docker daemon, no knowledge of the underlying remote machine should be required.
Subdomains, URLs and ports
Traffic is routed to services through HAProxy (which is, itself, a service). By looking in the docker-compose.yml file for HAProxy you can see the port numbers used by other services, each of which get a block of 10 ports by default (eg, 6010, 6020, 6030...). This allows for services which expose multiple ports to assign them consecutively and make them a little easier to remember during development.
When you start the HAProxy container it listens by default on ports 7080, 7443 and 7022 on the host machine for ports 80, 443 and 22 respectively. In order to route the traffic from those ports there is a config file in
haproxy/haproxy.cfg which usually matches based on subdomain. When developing a service you should claim a port range and add a rule here to route traffic then test it out by requesting a subdomain from one of the 7*** ports on your host system.
In order to setup a local wildcard domain for development on Linux you can use dnsmasq. The following will setup any domain ending in
.dev to point to 127.0.0.1 on Debian (and probably Ubuntu).
$ sudo apt-get install dnsmasq
Edit /etc/dnsmasq.conf and add "address=/dev/127.0.0.1"
$ sudo service dnsmasq restart
Edit /etc/dhcp/dhclient.conf and add "prepend domain-name-servers 127.0.0.1;".
$ sudo service network-manager restart
Now you can test HAProxy routes using https://subdomain.localhost.dev:port style urls (since the HAProxy rules are set to only match the first part of the domain).
Each service must provide scripts to help with maintenance. Currently, they are as follows:
- scripts/backup.sh DIRNAME - Backup a service. Accepts one argument, the directory on the host machine to put the backup files
- scripts/restore.sh DIRNAME - Restore a backup. Accepts one argument, the directory on the host where the backup files reside
- scripts/check_updates.sh - Checks running containers for available package updates - prints nothing if no updates available, prints one package per line otherwise - rebuilding the service's containers with
--no-cacheshould make this return an empty result again (see #Container guidelines)
Deployment to a VM
You can obtain a vagrant project describing the base system from:
git clone git://git.darkpeak.org/~caolan/darkpeak-host.git
This can then be provisioned and started using:
After it has started you can setup an SSH tunnel to access the docker daemon within using the
ssh_tunnel.sh script. It should then be possible to start services on the VM using the
docker-compose command as before:
DOCKER_HOST="tcp://localhost:4243" docker-compose up -d
Some services have a modified yaml file for deployment to a vagrant VM and you may have to specify that on the command line too:
DOCKER_HOST="tcp://localhost:4243" docker-compose -f docker-compose.vagrant.yml up -d
You can also inspect logs and start/stop services on the VM as usual by using the
DOCKER_HOST variable. This works with both docker and docker-compose commands.
In order to test out subdomain routing (like in the #Subdomains, URLs and ports section) you may want to add another entry to your
dnsmasq.conf, eg "address=/darkpeak.dev/192.168.33.10".
A service can authenticate Dark Peak users by communicating with the central LDAP directory.
The LDAP directory currently supports user authentication via an anonymous bind. Configure your service to talk to the directory at one of these URLs:
ldap://id.darkpeak.org:389/ # Unencrypted ldaps://id.darkpeak.org:636/ # Over TLS (recommended)
The "search base" to use for Dark Peak user accounts is:
With a search filter of:
Where "%u" is the username supplied by the user.