Developer mode

Stack overview

Backend

Callico's backend is built using Python and the Django framework. The database uses PostgreSQL. Asynchronous tasks are managed using a Celery queue on top of a Redis cache.

Frontend

Most of the frontend is generated by Django, supplemented by Django-Bulma and some Vanilla JS. Callico also has a few reactive components built with the Vue.js framework, compiled and served using Vue CLI.

Base setup

Once you have cloned the repository containing Callico's source code, in order to be able to develop and run a local instance of Callico, you need to:

1. Create a virtual environment, for example using virtualenvwrapper, and run the following command in your local cloned Callico Git repository: mkvirtualenv -a . callico
   • This will create a new virtual environment named callico, useful to isolate Python dependencies for each of your projects, which can be worked on using: workon callico
2. Install Callico as a package, in editable mode: pip install -e .

Python version requirement

• In order to develop on Callico, you need at least Python 3.11.

There are two ways you can launch a Callico instance for your development needs, in order to see the impact of your changes and check that everything still works as it should:

• Starting a local instance of Callico: this is the more complex option to set up, as it requires running multiple commands in different terminals, but this is also the responsive option. With a local instance, you can see your changes in real time, without needing to re-start the whole stack.
• Starting a dockerized instance of Callico: this is the easier mode, but it is not responsive to changes in real time like the local instance. If you update the code, you need to shut it down and build it again. You should use this option to test Callico's behavior once your code is ready, rather than to track the effects of your code as you write it.

Run a local instance

In order to get your local Callico instance up, you need to run a few commands in separate terminal windows/tabs to:

• start a database,
• get the frontend running, and
• launch the backend.

To have a fully functional instance that can also run tasks, such as importing data from a provider, you also need a Celery worker.

Start up the database

By default, local development or CI jobs will use a Postgresql database provided through Docker:

docker compose up postgres

Start up the frontend

The application uses Vue.js 3 to build complex UI components but also to serve Vanilla JS for simpler UI tasks.

The whole Javascript & CSS stack is available in the front directory.

When running a Callico instance locally for the first time, you have to install the necessary dependencies, by running this command from the front directory:

npm install

Next, you need to run the following command, to build styles & scripts and serve them to the backend:

npm run build

Afterwards, you can start the frontend so that it gets re-built when its code changes using the following command, in the front directory:

npm run watch

The following commands are available:

• npm run build builds the whole stack once in development mode. You need to run this command at least once to get styles & scripts running in Callico.
• npm run production does the same thing but in optimized mode (longer to build but smaller files).
• npm run serve builds demo pages (available in front/public) that allow developers to iterate quickly on the Vue.js components only, without using Callico.
• npm run watch builds the whole stack and awaits modifications to trigger immediately a new build. This is the recommended mode while working on an UI feature.

Vue.js components

Documentation

If you are adding/updating/deleting code that uses Vue.js frontend components, please read their dedicated documentation pages:

• Interactive Image,
• Carousel,
• Group Manager.

Demo pages

As mentioned above, the command npm run serve builds two small pages and opens a development server to demonstrate the various Vue.js components:

• a page serving the InteractiveImage component to manage elements at http://localhost:8080/interactive-image-elements/,
• a page serving the InteractiveImage, the Carousel and the GroupManager components at http://localhost:8080/group-elements/.

You can also customize the initial props of this demonstration by updating the front/public/*.js files.

Start up the backend

Lastly, you need to get the backend started by running the following command:

callico/manage.py runserver

You can now access your local Callico instance at localhost:8000.

Advanced settings

You can customize your local instance using the settings detailed in the application settings documentation.

Emails

If you want Django to be able to send emails, the EMAIL_URL and DEFAULT_FROM_EMAIL environment variables must be defined.

EMAIL_URL=smtp+tls://login:password@mails.server:587
DEFAULT_FROM_EMAIL=noreply@callico.com

Start up a Celery worker

If you want to be able to run asynchronous tasks on your local Callico instance, such as importing elements or exporting results, you need to have a Celery worker running.

To start a local Celery worker, you must have a Redis instance running:

docker compose up redis

You can also start a Flower interface to monitor your Celery cluster and its tasks, accessible at https://flower.dev.localhost/.

docker compose up flower

Then, you can use a make command to start the worker:

make worker

Your local Callico instance is now fully functional.

Docker

You can also run the full architecture using docker compose.

Before starting the project for the first time, or after an update, you need to:

(1) Run Callico containers with the build option:

docker compose up --build

(2) Once the images are built and the containers are running, run the migrate command to generate or migrate the database:

docker compose run callico django-admin migrate

(3) You should then create an admin user if none exists (using a command line form):

docker compose run callico django-admin createsuperuser

On a first run, you will also need to first log onto the local MinIO instance (user/password are both minio1234) and create the bucket callico-dev. Thanks to MinIO, media files will be stored on an online bucket instead of a local folder.

You can then access your application exposed through Traefik (with Teklia self-signed SSL cert) as https://callico.dev.localhost.

Debugging tools

In the development environment, you can install optional developer tools:

pip install ipython django-debug-toolbar django-extensions

IPython will give you a nicer shell with syntax highlighting, auto reloading and much more when using ./callico/manage.py shell.

Django Debug Toolbar provides you with a neat debug sidebar that will help diagnosing slow views or weird template bugs.

Django Extensions adds a lot of manage.py commands ; the most important one is ./callico/manage.py shell_plus which runs the usual shell but with all the available models pre-imported. You can add your own imports in a file at callico/base/local_settings.py. Here is an example that imports Django utils:

SHELL_PLUS_POST_IMPORTS = [
    ('django.db.models', ('Value', )),
    ('django.db.models.functions', '*'),
]