5 Minute Tutorial
Note
Recce Cloud is currently in private alpha and scheduled for general availability later this year. Sign up to the Recce newsletter to be notified, or email product@datarecce.io to join our design partnership program for early access.
Jaffle Shop is an example project officially provided by dbt-labs. This document uses jaffle_shop_duckdb to enable you to start using recce cloud from scratch within five minutes.
Clone Jaffle Shop to your Private Repositroy
- Create a private repository in your github account.
- Clone the Jaffle Shop dbt data project
- Change the remote url. Change the remote url to your repository.
- Push to your new created repository.
Authorize the repository to the Recce Cloud.
- Go to the recce cloud. If it is your first time to login, please click the Continue with Github and authorize your github account to the Recce Cloud GitHub App.
- Click the Install button to install Recce Cloud github app to your personal or organization account.
- In the app installation page in GitHub, authorize the new created repository to the app.
- Then it will show up all the authorized repositories.
Prepare the base environment
- Prepare virtual env
- Installation
- Provide additional environment to compare. Edit
./profiles.yml
to add one more target. - Add dbt packages for recce. Add
./packages.yml
and run - Prepare production environment
- Add
target-base/
folder to.gitignore
- Remove the existing github action workflow.
- Push to remote
Prepare the review state for the PR
As a PR author, you can prepare the recce review state and persist it in the recce cloud.
- Checkout a branch
- Prepare development environment. First, edit an existing model
./models/staging/stg_payments.sql
.
...
renamed as (
payment_method,
- -- `amount` is currently stored in cents, so we convert it to dollars
- amount / 100 as amount
+ amount
from source
)
run on development environment.
- Commit the change
- Create a pull request for this branch in your github repository.
- Prepare a github token in your account. You have to provide the
repo
permission. - Ensure you have configured these environment variables.
- Run recce server in the cloud mode Open the link http://0.0.0.0:8000
- Switch to the Query tab, add this query
Add the primary key
order_id
and click theRun Diff
button - Click the
+
button to add the query result to checklist - You can find that there are three checks in the Checks page
- Terminate the server. It would store the state to the recce cloud.
- In the GitHub PR page, we can find a failed check for this PR. This is because not all checks are approved.
Review the PR
As a PR author, you can review the PR by using the state stored in the recce cloud. If the checks are all good, you can approve them.
- Checkout the PR branch
- Ensure you have configured these environment variables.
- Run the recce server
- You can see the lineage diff and the checklist prepared by the PR author.
- Approve all the checks if everything looks good to you
- Go back to the GitHub PR page, you can find that the recce check is marked as passed.
Note
In this tutorial, we use duckdb as the warehouse, which is a file-based warehouse. The reviewer needs to have the same duckdb file to run the query.