Creating config
Developing a DipDup indexer begins with creating a YAML config file of a specific format. DipDup expect it to be located in the project root and named dipdup.yaml. However, you can provide any path with a -c CLI option.
Structure
Config consists of multiple top-level mappings. In the table below they are grouped by sections for convenience, but in the config file, they can be in any order. The only required section is Header.
| Header | spec_version* | DipDup project specification; currently "2.0" |
| package* | Python package name | |
| Inventory | database | Database configuration |
| contracts | Contracts that need to be indexed | |
| datasources | Datasources to gather information from | |
| Indexes | indexes | Index definitions |
| templates | Also index definitions, but with placeholders that make them reusable | |
| Hooks | hooks | Callbacks to run manually or by schedule |
| jobs | Schedules for hooks | |
| Integrations | hasura | Hasura GraphQL Engine configuration |
| sentry | Sentry configuration | |
| prometheus | Prometheus configuration | |
| Miscellanous | advanced | Tunables that affect framework behavior |
| custom | Mapping of user-defined values; neither typed nor validated | |
| logging | Configure logging verbosity |
Merging multiple files
DipDup allows you to customize the configuration for a specific environment or workflow. It works similarly to docker-compose anchors but only for top-level sections. If you want to override a nested property, you need to recreate a whole top-level section. To merge several DipDup config files, provide the -c command-line option multiple times:
dipdup -c dipdup.yaml -c configs/dipdup.sqlite.yaml runUse config export and config env commands to check the resulting config used by DipDup.
Environment variables
DipDup supports compose-style variable expansion with an optional default value. Use this feature to store sensitive data outside of the configuration file and make your app fully declarative. If a required variable is not set, DipDup will fail with an error. You can use these placeholders anywhere throughout the configuration file.
database:
kind: postgres
host: ${POSTGRES_HOST:-localhost}
password: ${POSTGRES_PASSWORD}There are multiple ways to pass environment variables to DipDup:
- Export them in the shell before running DipDup
- Create the env file and pass it to DipDup with the
-eCLI option
For every config file in the deploy project directory, DipDup will create a corresponding .env.default file with all the variables used in the config. Copy it, remove the .default suffix and fill in the values.
POSTGRES_HOST=localhost
POSTGRES_PASSWORD=You can use dipdup -e .env config export --unsafe command to ensure that env variables resolve as expected, but avoid sharing the output with third parties.