-
Notifications
You must be signed in to change notification settings - Fork 3.5k
CI: Speed up jobs and abtract documentation building #14985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
38fe5b9
50ba8e8
6304fc2
e278bc2
258e25c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # SPDX-FileCopyrightText: 2021 The Elixir Team | ||
|
|
||
| name: Build Elixir Documentation | ||
| description: Builds ExDoc and generates Elixir documentation | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| inputs: | ||
| otp_version: | ||
| description: The exact OTP version (major.minor[.patch]) | ||
| required: true | ||
|
|
||
| ex_doc_version: | ||
| description: ExDoc version (main or latest_stable) | ||
| required: true | ||
|
|
||
| warnings_as_errors: | ||
| description: Fails if ExDoc generates warnings | ||
| default: "false" | ||
|
|
||
| zip: | ||
| description: Whether to zip the docs | ||
| default: "false" | ||
|
|
||
| runs: | ||
| using: "composite" | ||
| steps: | ||
| - uses: erlef/setup-beam@e6d7c94229049569db56a7ad5a540c051a010af9 # v1.20.4 | ||
| with: | ||
| otp-version: ${{ inputs.otp_version }} | ||
| version-type: strict | ||
|
|
||
| - name: Compile Elixir | ||
| shell: bash | ||
| run: | | ||
| make compile | ||
| echo "$PWD/bin" >> $GITHUB_PATH | ||
|
|
||
| - name: Build info | ||
| shell: bash | ||
| run: bin/elixir --version | ||
|
|
||
| - name: Get ExDoc reference | ||
| shell: bash | ||
| run: | | ||
| if [ "${{ inputs.ex_doc_version }}" = "main" ]; then | ||
| ref=main | ||
| else | ||
| ref=v$(curl -s https://hex.pm/api/packages/ex_doc | jq --raw-output '.latest_stable_version') | ||
| fi | ||
|
|
||
| echo "EX_DOC_REF=$ref" >> $GITHUB_ENV | ||
|
|
||
| - name: Checkout ExDoc | ||
| uses: actions/checkout@1af3b93b6815bc44a9784bd300feb67ff0d1eeb3 # v6.0.0 | ||
| with: | ||
| repository: elixir-lang/ex_doc | ||
| ref: ${{ env.EX_DOC_REF }} | ||
| path: ex_doc | ||
|
|
||
| - name: Compile ExDoc | ||
| shell: bash | ||
| env: | ||
| LANG: C.UTF-8 | ||
| run: | | ||
| mv ex_doc ../ex_doc | ||
| cd ../ex_doc | ||
| ../elixir/bin/mix do local.rebar --force + local.hex --force + deps.get + compile && \ | ||
| echo "ExDoc successfully built." | ||
|
|
||
| - name: Set DOCS_OPTIONS | ||
| shell: bash | ||
| run: | | ||
| if [[ "${{ inputs.warnings_as_errors }}" == "true" ]]; then | ||
| echo "DOCS_OPTIONS=--warnings-as-errors" >> $GITHUB_ENV | ||
| else | ||
| echo "DOCS_OPTIONS=" >> $GITHUB_ENV | ||
| fi | ||
|
|
||
| - name: Build docs | ||
| shell: bash | ||
| env: | ||
| LANG: C.UTF-8 | ||
| run: | | ||
| cd ../elixir | ||
| git fetch --tags | ||
| make ${{ inputs.zip == 'true' && 'Docs.zip' || 'docs' }} && \ | ||
| if [[ "${{ inputs.warnings_as_errors }}" == "true" ]]; then | ||
| echo "Docs successfully generated without warnings." | ||
| else | ||
| echo "Docs successfully generated." | ||
| fi |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,15 +2,27 @@ | |
| # SPDX-FileCopyrightText: 2021 The Elixir Team | ||
| # SPDX-FileCopyrightText: 2012 Plataformatec | ||
|
|
||
| name: CI | ||
| name: Elixir Building and Testing | ||
|
|
||
| on: | ||
| push: | ||
| paths-ignore: | ||
| - "lib/**/*.md" | ||
| paths: &paths-filter | ||
| - .github/workflows/ci.yml | ||
| - lib/*/test/fixtures/**/**.md | ||
|
|
||
| # Ignore | ||
| - "!*.{md,yml,jsonc}" | ||
| - "!.git*" | ||
| - "!.github/workflows/**/*" | ||
| - "!.ort/**/*" | ||
| - "!lib/*/pages/**/*.md" | ||
| - "!LICENSES/*" | ||
| - "!man/*" | ||
|
|
||
| pull_request: | ||
| paths-ignore: | ||
| - "lib/**/*.md" | ||
| paths: *paths-filter | ||
|
|
||
| workflow_dispatch: | ||
|
|
||
| env: | ||
| ELIXIR_ASSERT_TIMEOUT: 2000 | ||
|
|
@@ -27,79 +39,94 @@ jobs: | |
|
|
||
| strategy: | ||
| fail-fast: false | ||
|
|
||
| matrix: | ||
| include: | ||
| # NOTE: When updating latest Erlang/OTP version, also update it in | ||
| # .github/workflows/documentation.yml | ||
| - otp_version: "28.1" | ||
| erlc_opts: "warnings_as_errors" | ||
| deterministic: true | ||
| skip_tests: true | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't want to add new CI entries exactly because for building docs we already have to compile and what not. And doing everything twice is wasteful.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So to be clear, we should still have a single entry that compiles, run docs, and test for deterministic, as it was done before. :) |
||
|
|
||
| - otp_version: "28.1" | ||
| erlc_opts: "warnings_as_errors" | ||
| coverage: true | ||
| skip_tests: true | ||
|
|
||
| - otp_version: "28.1" | ||
| otp_latest: true | ||
| erlc_opts: "warnings_as_errors" | ||
|
|
||
| - otp_version: "27.3" | ||
| erlc_opts: "warnings_as_errors" | ||
|
|
||
| - otp_version: "27.0" | ||
| erlc_opts: "warnings_as_errors" | ||
|
|
||
| - otp_version: "26.0" | ||
|
|
||
| - otp_version: master | ||
| development: true | ||
|
|
||
| - otp_version: maint | ||
| development: true | ||
|
|
||
| # Earlier Erlang/OTP versions ignored compiler directives | ||
| # when using warnings_as_errors. So we only set ERLC_OPTS | ||
| # from Erlang/OTP 27+. | ||
| env: | ||
| ERLC_OPTS: ${{ matrix.erlc_opts || '' }} | ||
| ERLC_OPTS: ${{ matrix.erlc_opts }} | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@1af3b93b6815bc44a9784bd300feb67ff0d1eeb3 # v6.0.0 | ||
| with: | ||
| fetch-depth: 50 | ||
| - uses: erlef/setup-beam@e6d7c94229049569db56a7ad5a540c051a010af9 # v1.20.4 | ||
| with: | ||
| otp-version: ${{ matrix.otp_version }} | ||
|
|
||
| - name: Set ERL_COMPILER_OPTIONS | ||
| if: ${{ matrix.deterministic }} | ||
| run: echo "ERL_COMPILER_OPTIONS=deterministic" >> $GITHUB_ENV | ||
|
|
||
| - name: Compile Elixir | ||
| run: | | ||
| make compile | ||
| echo "$PWD/bin" >> $GITHUB_PATH | ||
| - name: Build info | ||
| run: bin/elixir --version | ||
|
|
||
| - name: Check format | ||
| run: make test_formatted && echo "All Elixir source code files are properly formatted." | ||
| if: ${{ !matrix.skip_tests }} | ||
| run: | | ||
| make test_formatted && \ | ||
| echo "All Elixir source code files are properly formatted." | ||
| - name: Erlang test suite | ||
| if: ${{ !matrix.skip_tests }} | ||
| run: make test_erlang | ||
| continue-on-error: ${{ matrix.development }} | ||
| continue-on-error: ${{ matrix.development == true }} | ||
|
|
||
| - name: Elixir test suite | ||
| if: ${{ matrix.coverage || !matrix.skip_tests }} | ||
| run: make test_elixir | ||
| continue-on-error: ${{ matrix.development }} | ||
| continue-on-error: ${{ matrix.development == true }} | ||
| env: | ||
| COVER: "${{ matrix.coverage }}" | ||
| - name: "Calculate Coverage" | ||
| run: make cover | tee "$GITHUB_STEP_SUMMARY" | ||
| if: "${{ matrix.coverage }}" | ||
| - name: Build docs (ExDoc main) | ||
| if: ${{ matrix.otp_latest }} | ||
| run: | | ||
| cd .. | ||
| git clone https://github.com/elixir-lang/ex_doc.git --depth 1 | ||
| cd ex_doc | ||
| ../elixir/bin/mix do local.rebar --force + local.hex --force + deps.get + compile | ||
| cd ../elixir/ | ||
| git fetch --tags | ||
| DOCS_OPTIONS="--warnings-as-errors" make docs | ||
| COVER: ${{ matrix.coverage }} | ||
|
|
||
| - name: Check reproducible builds | ||
| if: ${{ matrix.deterministic }} | ||
| run: | | ||
| rm -rf .git | ||
| # Recompile System without .git | ||
| cd lib/elixir && ../../bin/elixirc -o ebin lib/system.ex && cd - | ||
| taskset 1 make check_reproducible | ||
| - name: "Upload Coverage Artifact" | ||
| if: "${{ matrix.coverage }}" | ||
| - name: Calculate Coverage | ||
| if: ${{ matrix.coverage }} | ||
| run: make cover | tee "$GITHUB_STEP_SUMMARY" | ||
|
|
||
| - name: Upload Coverage Artifact | ||
| if: ${{ matrix.coverage }} | ||
| uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0 | ||
| with: | ||
| name: TestCoverage | ||
|
|
@@ -119,22 +146,29 @@ jobs: | |
| steps: | ||
| - name: Configure Git | ||
| run: git config --global core.autocrlf input | ||
|
|
||
| - uses: actions/checkout@1af3b93b6815bc44a9784bd300feb67ff0d1eeb3 # v6.0.0 | ||
| with: | ||
| fetch-depth: 50 | ||
|
|
||
| - uses: erlef/setup-beam@e6d7c94229049569db56a7ad5a540c051a010af9 # v1.20.4 | ||
| with: | ||
| otp-version: ${{ matrix.otp_version }} | ||
|
|
||
| - name: Compile Elixir | ||
| run: | | ||
| Remove-Item -Recurse -Force '.git' | ||
| make compile | ||
| - name: Build info | ||
| run: bin/elixir --version | ||
|
|
||
| - name: Check format | ||
| run: make test_formatted && echo "All Elixir source code files are properly formatted." | ||
| run: | | ||
| make test_formatted && \ | ||
| echo "All Elixir source code files are properly formatted." | ||
| - name: Erlang test suite | ||
| run: make test_erlang | ||
|
|
||
| - name: Elixir test suite | ||
| run: | | ||
| Remove-Item 'c:/Windows/System32/drivers/etc/hosts' | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # SPDX-FileCopyrightText: 2021 The Elixir Team | ||
|
|
||
| name: Elixir Documentation | ||
|
|
||
| on: | ||
| push: | ||
| paths: &paths-filter | ||
| - .github/workflows/documentation.yml | ||
| - .github/workflows/build_docs/**/* | ||
| - CHANGELOG.md | ||
| - Makefile | ||
| - VERSION | ||
| - lib/**/* | ||
|
|
||
| # Ignore | ||
| - "!lib/*/test/**/*" | ||
|
|
||
| pull_request: | ||
| paths: *paths-filter | ||
|
|
||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| build_docs: | ||
| name: Build docs | ||
| runs-on: ubuntu-24.04 | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@1af3b93b6815bc44a9784bd300feb67ff0d1eeb3 # v6.0.0 | ||
|
|
||
| - name: Build docs | ||
| uses: ./.github/workflows/build_docs | ||
| with: | ||
| otp_version: "28.1" | ||
| ex_doc_version: "main" | ||
| warnings_as_errors: true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please revert the precise "paths"? Those other files change rarely and one day we may add a markdown fixture that actually changes the test, and we will spend a long time without noticing it. Unless the files are common and change frequently and they do non't need to run (the case for
lib/*/pages/**/*.md, we shouldn't worry about excluding them).