Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env.development
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ SOURCEBOT_TELEMETRY_DISABLED=true # Disables telemetry collection
# NEXT_PUBLIC_SOURCEBOT_VERSION=

# CONFIG_MAX_REPOS_NO_TOKEN=
# NODE_ENV=
NODE_ENV=development
# SOURCEBOT_TENANCY_MODE=single

# NEXT_PUBLIC_SOURCEBOT_CLOUD_ENVIRONMENT=
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## Added
- [Experimental][Sourcebot EE] Added GitLab permission syncing. [#585](https://github.com/sourcebot-dev/sourcebot/pull/585)
- [Sourcebot EE] Added external identity provider config and support for multiple accounts. [#595](https://github.com/sourcebot-dev/sourcebot/pull/595)
- Added ability to configure environment variables from the config. [#597](https://github.com/sourcebot-dev/sourcebot/pull/597)

### Fixed
- [ask sb] Fixed issue where reasoning tokens would appear in `text` content for openai compatible models. [#582](https://github.com/sourcebot-dev/sourcebot/pull/582)
Expand Down
16 changes: 0 additions & 16 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -42,16 +42,10 @@ COPY package.json yarn.lock* .yarnrc.yml ./
COPY .yarn ./.yarn
COPY ./packages/db ./packages/db
COPY ./packages/schemas ./packages/schemas
COPY ./packages/crypto ./packages/crypto
COPY ./packages/error ./packages/error
COPY ./packages/logger ./packages/logger
COPY ./packages/shared ./packages/shared

RUN yarn workspace @sourcebot/db install
RUN yarn workspace @sourcebot/schemas install
RUN yarn workspace @sourcebot/crypto install
RUN yarn workspace @sourcebot/error install
RUN yarn workspace @sourcebot/logger install
RUN yarn workspace @sourcebot/shared install
# ------------------------------------

Expand Down Expand Up @@ -97,9 +91,6 @@ COPY ./packages/web ./packages/web
COPY --from=shared-libs-builder /app/node_modules ./node_modules
COPY --from=shared-libs-builder /app/packages/db ./packages/db
COPY --from=shared-libs-builder /app/packages/schemas ./packages/schemas
COPY --from=shared-libs-builder /app/packages/crypto ./packages/crypto
COPY --from=shared-libs-builder /app/packages/error ./packages/error
COPY --from=shared-libs-builder /app/packages/logger ./packages/logger
COPY --from=shared-libs-builder /app/packages/shared ./packages/shared

# Fixes arm64 timeouts
Expand Down Expand Up @@ -138,9 +129,6 @@ COPY ./packages/backend ./packages/backend
COPY --from=shared-libs-builder /app/node_modules ./node_modules
COPY --from=shared-libs-builder /app/packages/db ./packages/db
COPY --from=shared-libs-builder /app/packages/schemas ./packages/schemas
COPY --from=shared-libs-builder /app/packages/crypto ./packages/crypto
COPY --from=shared-libs-builder /app/packages/error ./packages/error
COPY --from=shared-libs-builder /app/packages/logger ./packages/logger
COPY --from=shared-libs-builder /app/packages/shared ./packages/shared
RUN yarn workspace @sourcebot/backend install
RUN yarn workspace @sourcebot/backend build
Expand Down Expand Up @@ -185,7 +173,6 @@ ENV DATA_DIR=/data
ENV DATA_CACHE_DIR=$DATA_DIR/.sourcebot
ENV DATABASE_DATA_DIR=$DATA_CACHE_DIR/db
ENV REDIS_DATA_DIR=$DATA_CACHE_DIR/redis
ENV REDIS_URL="redis://localhost:6379"
ENV SRC_TENANT_ENFORCEMENT_MODE=strict
ENV SOURCEBOT_PUBLIC_KEY_PATH=/app/public.pem

Expand Down Expand Up @@ -225,9 +212,6 @@ COPY --from=backend-builder /app/packages/backend ./packages/backend
COPY --from=shared-libs-builder /app/node_modules ./node_modules
COPY --from=shared-libs-builder /app/packages/db ./packages/db
COPY --from=shared-libs-builder /app/packages/schemas ./packages/schemas
COPY --from=shared-libs-builder /app/packages/crypto ./packages/crypto
COPY --from=shared-libs-builder /app/packages/error ./packages/error
COPY --from=shared-libs-builder /app/packages/logger ./packages/logger
COPY --from=shared-libs-builder /app/packages/shared ./packages/shared

# Configure dependencies
Expand Down
4 changes: 0 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,6 @@ clean:
packages/db/dist \
packages/schemas/node_modules \
packages/schemas/dist \
packages/crypto/node_modules \
packages/crypto/dist \
packages/error/node_modules \
packages/error/dist \
packages/mcp/node_modules \
packages/mcp/dist \
packages/shared/node_modules \
Expand Down
103 changes: 103 additions & 0 deletions docs/docs/configuration/config-file.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ title: Config File
sidebarTitle: Config file
---

import ConfigSchema from '/snippets/schemas/v3/index.schema.mdx'
import EnvironmentOverridesSchema from '/snippets/schemas/v3/environmentOverrides.schema.mdx'

When self-hosting Sourcebot, you **must** provide it a config file. This is done by defining a config file in a volume that's mounted to Sourcebot, and providing the path to this
file in the `CONFIG_PATH` environment variable. For example:

Expand Down Expand Up @@ -49,3 +52,103 @@ The following are settings that can be provided in your config file to modify So
| `enablePublicAccess` **(deprecated)** | boolean | false | — | Use the `FORCE_ENABLE_ANONYMOUS_ACCESS` environment variable instead. |
| `experiment_repoDrivenPermissionSyncIntervalMs` | number | 24 hours | 1 | Interval at which the repo permission syncer should run. |
| `experiment_userDrivenPermissionSyncIntervalMs` | number | 24 hours | 1 | Interval at which the user permission syncer should run. |

# Tokens

Tokens are used to securely pass secrets to Sourcebot in a config file. They are used in various places, including connections, language model providers, auth providers, etc. Tokens can be passed as either environment variables or Google Cloud secrets:

<AccordionGroup>
<Accordion title="Environment Variables">
```json
{
"token": {
"env": "TOKEN_NAME"
}
}
```
</Accordion>
<Accordion title="Google Cloud Secrets">
```json
{
"token": {
"googleCloudSecret": "projects/<project-id>/secrets/<secret-name>/versions/<version-id>"
}
}
```
</Accordion>
</AccordionGroup>

# Overriding environment variables from the config

You can override / set environment variables from the config file by using the `environmentOverrides` property. Overrides can be of type `string`, `number`, `boolean`, or a [token](/docs/configuration/config-file#tokens). Tokens are useful when you want to configure a environment variable using a Google Cloud Secret or other supported secret management service.

<AccordionGroup>
<Accordion title="Token">
```jsonc
{
"environmentOverrides": {
"DATABASE_URL": {
"type": "token",
"value": {
"googleCloudSecret": "projects/<id>/secrets/postgres-connection-string/versions/latest"
}
},
"REDIS_URL": {
"type": "token",
"value": {
"googleCloudSecret": "projects/<id>/secrets/redis-connection-string/versions/latest"
}
}
},
}
```
</Accordion>

<Accordion title="String">
```jsonc
{
"environmentOverrides": {
"EMAIL_FROM_ADDRESS": {
"type": "string",
"value": "hello@sourcebot.dev"
}
}
}
```
</Accordion>

<Accordion title="Number">
```jsonc
{
"environmentOverrides": {
"SOURCEBOT_CHAT_MODEL_TEMPERATURE": {
"type": "number",
"value": 0.5
}
}
}
```
</Accordion>

<Accordion title="Boolean">
```jsonc
{
"environmentOverrides": {
"SOURCEBOT_TELEMETRY_DISABLED": {
"type": "boolean",
"value": false
}
}
}
```
</Accordion>
</AccordionGroup>


**Note:** Overrides are **not** set as system environment variables, and instead are resolved at runtime on startup and stored in memory.

<Accordion title="Schema reference">
[schemas/v3/environmentOverrides.json](https://github.com/sourcebot-dev/sourcebot/blob/main/schemas/v3/environmentOverrides.json)

<EnvironmentOverridesSchema />
</Accordion>
4 changes: 3 additions & 1 deletion docs/docs/configuration/environment-variables.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
title: Environment variables
sidebarTitle: Environment variables
mode: "wide"
---

<Note>This page provides a detailed reference of all environment variables supported by Sourcebot. If you're just looking to get up and running, we recommend starting with the [deployment guide](/docs/deployment-guide) instead.</Note>
Expand Down Expand Up @@ -71,3 +70,6 @@ The following environment variables allow you to configure your Sourcebot deploy
| `REVIEW_AGENT_LOGGING_ENABLED` | `true` | <p>Enables/disables logging for the review agent. Logs are saved in `DATA_CACHE_DIR/review-agent`</p> |
| `REVIEW_AGENT_REVIEW_COMMAND` | `review` | <p>The command used to trigger a code review by the review agent.</p> |

### Overriding environment variables from the config

You can override environment variables from the config file by using the `environmentOverrides` property. See [this doc](/docs/configuration/config-file#overriding-environment-variables-from-the-config) for more info.
7 changes: 2 additions & 5 deletions docs/docs/configuration/idp.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,6 @@ External identity providers can be used for [authentication](/docs/configuration
"provider": "github",
"purpose": "account_linking",
"accountLinkingRequired": true,
/*
Secrets are provided through environment variables. Set the secret into
an env var and provide the name here to tell Sourcebot where to get
the value
*/
"clientId": {
"env": "GITHUB_IDENTITY_PROVIDER_CLIENT_ID"
},
Expand All @@ -45,6 +40,8 @@ the value
}
```

Secret values (such as `clientId` and `clientSecret`) can be provided as environment variables or Google Cloud secrets via [tokens](/docs/configuration/config-file#tokens).

# Supported External Identity Providers

Sourcebot uses [Auth.js](https://authjs.dev/) to connect to external identity providers. If there's a provider supported by Auth.js that you don't see below, please submit a
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/connections/ado-cloud.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ If you're not familiar with Sourcebot [connections](/docs/connections/overview),
Azure Devops Cloud requires you to provide a PAT in order to index your repositories. To learn how to create PAT, check out the [Azure Devops docs](https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows).
Sourcebot needs the `Read` access for the `Code` scope in order to find and clone your repos.

Next, provide the access token via an environment variable which is referenced in the `token` property:
Next, provide the access [token](/docs/configuration/config-file#tokens) via an environment variable which is referenced in the `token` property:

<Tabs>
<Tab title="Environment Variable">
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/connections/ado-server.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ If you're not familiar with Sourcebot [connections](/docs/connections/overview),
Azure Devops Server requires you to provide a PAT in order to index your repositories. To learn how to create PAT, check out the [Azure Devops docs](https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows).
Sourcebot needs the `Read` access for the `Code` scope in order to find and clone your repos.

Next, provide the access token via an environment variable which is referenced in the `token` property:
Next, provide the access [token](/docs/configuration/config-file#tokens) via an environment variable which is referenced in the `token` property:

<Tabs>
<Tab title="Environment Variable">
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/connections/bitbucket-cloud.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ If you're not familiar with Sourcebot [connections](/docs/connections/overview),

## Authenticating with Bitbucket Cloud

In order to index private repositories, you'll need to provide authentication credentials. You can do this using an `App Password` or an `Access Token`
In order to index private repositories, you'll need to provide authentication credentials via a [token](/docs/configuration/config-file#tokens). You can do this using an `App Password` or an `Access Token`

<Tabs>
<Tab title="App Password">
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/connections/bitbucket-data-center.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ If you're not familiar with Sourcebot [connections](/docs/connections/overview),

## Authenticating with Bitbucket Data Center

In order to index private repositories, you'll need to provide an access token to Sourcebot.
In order to index private repositories, you'll need to provide an access token to Sourcebot via a [token](/docs/configuration/config-file#tokens).

Create an access token for the desired scope (repo, project, or workspace). Visit the official [Bitbucket Data Center docs](https://confluence.atlassian.com/bitbucketserver/http-access-tokens-939515499.html)
for more info.
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/connections/gitea.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ In order to index private repositories, you'll need to generate a Gitea access t

![Gitea Access token creation](/images/gitea_pat_creation.png)

Next, provide the access token via an environment variable which is referenced in the `token` property:
Next, provide the access token via an environment variable [token](/docs/configuration/config-file#tokens) which is referenced in the `token` property:

<Tabs>
<Tab title="Environment Variable">
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/connections/github.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ In order to index private repositories, you'll need to generate a access token a
</Accordion>
</AccordionGroup>

Next, provide the access token via an environment variable which is referenced in the `token` property:
Next, provide the access token via an environment variable [token](/docs/configuration/config-file#tokens) which is referenced in the `token` property:

<Tabs>
<Tab title="Environment Variable">
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/connections/gitlab.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ In order to index private projects, you'll need to generate a GitLab Personal Ac

![GitLab PAT Scope](/images/gitlab_pat_scopes.png)

Next, provide the PAT via an environment variable which is referenced in the `token` property:
Next, provide the PAT via an environment variable [token](/docs/configuration/config-file#tokens) which is referenced in the `token` property:

<Tabs>
<Tab title="Environment Variable">
Expand Down
Loading
Loading