iHack, therefore iBlog

LightDash is a super-cool Open Source business intelligence tool built on top of DBT (which I think of as node for SQL). While it is distributed as open source, the usual way to deploy it locally is by simply running a docker container.

If you want to actually built lightdash directly from source yourself, you need to follow the instructions under CONTRIBUTING. However, what was written there (as of November 11, 2021) did not quite work for me, so here are my workarounds.

I will also file this as a GitHub issue, and they are super-responsive so hopefully this page will be obsolete soon!

A. Using ODBC from a non-standard Homebrew location

One of the few native drivers that most scripting languages need to build is for ODBC. This leads to the infamous “missing <sql.h>” error, which bit me here, perhaps since I have a non-standard per-user Homebrew installation. Here’s what ended up working for me:

CELLAR_ODBC=`brew info unixodbc | grep Cellar | awk '{print $1}'`
LIB_ODBC=$CELLAR_ODBC/lib 
SRC_ODBC=$CELLAR_ODBC/include
export C_INCLUDE_PATH=$C_INCLUDE_PATH:$SRC_ODBC
export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:$SRC_ODBC
export LIBRARY_PATH=$LIBRARY_PATH:$LIB_ODBC
yarn install

B. Configuring LightDash

It wasn’t obvious to me how to do this. The proper sequence appears to be:

1. Add your DBT configuration to lightdash.yml file as explained in the documentation
2. Make up a LIGHTDASH_SECRET then export as an environment variable:.
   1. $ export LIGHTDASH_SECRET=MySpecialSecret
3. You must use the same secret every time you run LightDash, or you won’t be able to access your previous data!
4. Export that file to Postgres:
   1.  export LIGHTDASH_CONFIG_FILE=${PWD}/lightdash.yml

C. Specify your Postgres envars before launching Docker

The CONTRIBUTING file seemed to specify it the other way round, but I found it worked if I did this first:

export LIGHTDASH_SECRET=MySpecialSecretexport LIGHTDASH_CONFIG_FILE=${PWD}/lightdash.yml
export PGHOST=127.0.0.1
export PGPORT=5432
export PGDATABASE=postgres
export PGUSER=postgres
export PGPASSWORD=mysecretpassword
docker run --name lightdash-db -p "5432:5432" -e POSTGRES_PASSWORD=mysecretpassword -d postgres

Again, it does not seem to matter which password you choose as long as you are consistent

D. Use Yarn to build and run LightDash

• yarn workspace backend migrate
• yarn common-build && yarn dev

Tagged: data, Open Source, sql