This page attempts to introduce developers to the QMK Compiler. It does not go into nitty gritty details- for that you should read code. What this will give you is a framework to hang your understanding on as you read the code.
The QMK Compile API consists of a few movings parts:
API Clients interact exclusively with the API service. This is where they submit jobs, check status, and download results. The API service inserts compile jobs into Redis Queue and checks both RQ and S3 for the results of those jobs.
Workers fetch new compile jobs from RQ, compile them, and then upload the source and the binary to an S3 compatible storage engine.
QMK Compiler Workers are responsible for doing the actual building. When a worker pulls a job from RQ it does several things to complete that job:
Make a fresh qmk_firmware checkout
Use the supplied layers and keyboard metadata to build a keymap.c
Build the firmware
Zip a copy of the source
Upload the firmware, source zip, and a metadata file to S3.
Report the status of the job to RQ
The API service is a relatively simple Flask application. There are a few main views you should understand.
This is the main entrypoint for the API. A client's interaction starts here. The client POST's a JSON document describing their keyboard, and the API does some (very) basic validation of that JSON before submitting the compile job.