Skip to content

Conversation

@mfyll
Copy link

@mfyll mfyll commented Oct 29, 2025

Add OpenReports JSON Import Parser

Description

Adds support for importing scans in OpenReports.io format. Currently only statnett operator outputs trivy scans in this format, but being an open standard backed by e.g kyverno, more tools could adopt it.

More information can be found here: OpenReports.io

Steps to get the relevant file:

kubectl get reports -ojson -A > reports.json

Features:

  • Parses single reports and Kubernetes List objects
  • Extracts CVE IDs, component versions, and fix availability
  • Maps severity levels and creates proper service identifiers
  • Handles vulnerability metadata and tagging

Test results

Added unit tests covering empty results, single reports, list format, and parser metadata. All tests pass with sample JSON files included.

I have also tested this by importing the resulting JSON file in the DefectDojo GUI and this works.

Documentation

Parser follows standard DefectDojo interface with inline documentation and sample files.

Checklist

  • Rebased against latest dev
  • Submitted against dev branch
  • Flake8 and Python 3.12 compliant
  • Unit tests added

Copy link
Member

@valentijnscholten valentijnscholten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! It looks quite good already. Some questions:

  • Can you add at least one vulnerability for which the policy field is not a CVE?
  • Could you look at the hash_code configuration for deduplication? It might be good to check wha the best fit ir or if there's a field that we can use as a value for unique_id_from_tool.

Copy link
Contributor

@manuel-sommer manuel-sommer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also add a documentation file for the parser.

@github-actions github-actions bot added settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR docs labels Oct 30, 2025
@mfyll mfyll force-pushed the add-openreports-import branch from 8cbb5ca to 7647272 Compare October 30, 2025 10:15
@valentijnscholten valentijnscholten added this to the 2.52.0 milestone Oct 30, 2025
@valentijnscholten
Copy link
Member

I think also if this is a generic report format similar to SARIF, the openreports parser should do something similar where the actual report type / scanner type ends up in the test name in Defect Dojo.
I believe something happens here:

test = ParserTest(
name=run["tool"]["driver"]["name"],
parser_type=run["tool"]["driver"]["name"],
version=run["tool"]["driver"].get("version"),
)

@mfyll mfyll force-pushed the add-openreports-import branch from 01dc290 to 53ef8bb Compare October 31, 2025 07:48
Copy link
Member

@valentijnscholten valentijnscholten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for extending the PR. The values for the scanner names seem a bit generic and may change in the future if more operators/scanners will use this output format, but that's outside our control.

Copy link
Member

@valentijnscholten valentijnscholten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, forgot that there were some small issues to address.

@valentijnscholten valentijnscholten modified the milestones: 2.52.0, 2.53.0 Nov 3, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 5, 2025

Conflicts have been resolved. A maintainer will review the pull request shortly.

@valentijnscholten valentijnscholten marked this pull request as draft November 14, 2025 16:37
@valentijnscholten
Copy link
Member

Changing to draft status until it's ready.

@mfyll mfyll marked this pull request as ready for review November 18, 2025 08:58
@dryrunsecurity
Copy link

DryRun Security

This pull request introduces a parser (OpenreportsParser) that reads uploaded scan files into memory without size limits, which could allow a maliciously large upload to exhaust process memory and cause a Denial of Service. Consider adding file size checks, streaming/iterative processing, or explicit limits and error handling to mitigate the risk.

Resource Exhaustion (Denial of Service) in dojo/tools/openreports/parser.py
Vulnerability Resource Exhaustion (Denial of Service)
Description The OpenreportsParser reads the entire uploaded scan file into memory using scan_file.read() without any explicit size limitations within the parser. A malicious user could upload an extremely large file, causing the application process to consume excessive memory, potentially leading to a Denial of Service (DoS) by crashing the process.

scan_data = scan_file.read()
try:
data = json.loads(str(scan_data, "utf-8"))


All finding details can be found in the DryRun Security Dashboard.

@valentijnscholten valentijnscholten marked this pull request as draft November 19, 2025 08:27
@valentijnscholten
Copy link
Member

Marked as draft for now. Can you fix the failing tests @mfyll

@mfyll
Copy link
Author

mfyll commented Nov 19, 2025

Marked as draft for now. Can you fix the failing tests @mfyll

Should be OK now.

@mfyll mfyll marked this pull request as ready for review November 19, 2025 09:25
@dryrunsecurity
Copy link

dryrunsecurity bot commented Nov 19, 2025

DryRun Security

This pull request adds or modifies an OpenreportsParser that reads the entire uploaded report file into memory via scan_file.read() without size validation, which could allow a maliciously large upload to exhaust memory and cause a Denial of Service. Consider validating or limiting file size and streaming processing to avoid loading the whole file into memory.

Resource Exhaustion (Denial of Service) in dojo/tools/openreports/parser.py
Vulnerability Resource Exhaustion (Denial of Service)
Description The OpenreportsParser reads the entire uploaded report file into memory using scan_file.read() without any explicit size validation. While Django, the underlying framework, spools large files (by default, > 2.5MB) to disk, the parser's subsequent read() operation will still attempt to load the entire content of this potentially very large file from disk back into the application's memory. A malicious actor could upload an extremely large file, causing the application to consume excessive memory, leading to a Denial of Service (DoS) by crashing the process.

scan_data = scan_file.read()
try:
data = json.loads(str(scan_data, "utf-8"))


All finding details can be found in the DryRun Security Dashboard.

@valentijnscholten
Copy link
Member

Some failures. Please note the folder location of the parser has changed.

2025-11-19T16:31:54.8042342Z uwsgi-1  | FAIL: test_file_existence (unittests.test_parsers.TestParsers.test_file_existence) (parser='openreports', category='docs')
2025-11-19T16:31:54.8042496Z uwsgi-1  | ----------------------------------------------------------------------
2025-11-19T16:31:54.8042605Z uwsgi-1  | Traceback (most recent call last):
2025-11-19T16:31:54.8042816Z uwsgi-1  |   File "/app/unittests/test_parsers.py", line 33, in test_file_existence
2025-11-19T16:31:54.8042906Z uwsgi-1  |     self.assertTrue(
2025-11-19T16:31:54.8042989Z uwsgi-1  |     ~~~~~~~~~~~~~~~^
2025-11-19T16:31:54.8043089Z uwsgi-1  |         Path(doc_file).is_file(),
2025-11-19T16:31:54.8043183Z uwsgi-1  |         ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-11-19T16:31:54.8043388Z uwsgi-1  |         f"Documentation file '{doc_file}' is missing or using different name",
2025-11-19T16:31:54.8043514Z uwsgi-1  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-11-19T16:31:54.8043596Z uwsgi-1  |                     )
2025-11-19T16:31:54.8043671Z uwsgi-1  |                     ^
2025-11-19T16:31:54.8044152Z uwsgi-1  | AssertionError: False is not true : Documentation file '/app/docs/content/supported_tools/parsers/file/openreports.md' is missing or using different name
2025-11-19T16:31:54.8044228Z uwsgi-1  | 
2025-11-19T16:31:54.8044354Z uwsgi-1  | ======================================================================
2025-11-19T16:31:54.8044925Z uwsgi-1  | FAIL: test_get_tests_multiple_sources (unittests.tools.test_openreports_parser.TestOpenreportsParser.test_get_tests_multiple_sources)
2025-11-19T16:31:54.8045087Z uwsgi-1  | ----------------------------------------------------------------------
2025-11-19T16:31:54.8045242Z uwsgi-1  | Traceback (most recent call last):
2025-11-19T16:31:54.8045541Z uwsgi-1  |   File "/app/unittests/tools/test_openreports_parser.py", line 154, in test_get_tests_multiple_sources
2025-11-19T16:31:54.8045706Z uwsgi-1  |     self.assertEqual(2, len(image_scanner_test.findings))
2025-11-19T16:31:54.8045831Z uwsgi-1  |     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-11-19T16:31:54.8045973Z uwsgi-1  | AssertionError: 2 != 0
2025-11-19T16:31:54.8046049Z uwsgi-1  | 
2025-11-19T16:31:54.8046171Z uwsgi-1  | ======================================================================
2025-11-19T16:31:54.8046559Z uwsgi-1  | FAIL: test_get_tests_single_source (unittests.tools.test_openreports_parser.TestOpenreportsParser.test_get_tests_single_source)
2025-11-19T16:31:54.8046715Z uwsgi-1  | ----------------------------------------------------------------------
2025-11-19T16:31:54.8046821Z uwsgi-1  | Traceback (most recent call last):
2025-11-19T16:31:54.8047110Z uwsgi-1  |   File "/app/unittests/tools/test_openreports_parser.py", line 130, in test_get_tests_single_source
2025-11-19T16:31:54.8047272Z uwsgi-1  |     self.assertEqual(2, len(image_scanner_test.findings))
2025-11-19T16:31:54.8047396Z uwsgi-1  |     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-11-19T16:31:54.8047487Z uwsgi-1  | AssertionError: 2 != 0
2025-11-19T16:31:54.8047559Z uwsgi-1  | 
2025-11-19T16:31:54.8047683Z uwsgi-1  | ======================================================================
2025-11-19T16:31:54.8047995Z uwsgi-1  | FAIL: test_list_format (unittests.tools.test_openreports_parser.TestOpenreportsParser.test_list_format)
2025-11-19T16:31:54.8048315Z uwsgi-1  | ----------------------------------------------------------------------
2025-11-19T16:31:54.8048425Z uwsgi-1  | Traceback (most recent call last):
2025-11-19T16:31:54.8048676Z uwsgi-1  |   File "/app/unittests/tools/test_openreports_parser.py", line 81, in test_list_format
2025-11-19T16:31:54.8048790Z uwsgi-1  |     self.assertEqual(len(findings), 3)
2025-11-19T16:31:54.8048888Z uwsgi-1  |     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
2025-11-19T16:31:54.8048979Z uwsgi-1  | AssertionError: 0 != 3
2025-11-19T16:31:54.8049053Z uwsgi-1  | 
2025-11-19T16:31:54.8049172Z uwsgi-1  | ======================================================================
2025-11-19T16:31:54.8049497Z uwsgi-1  | FAIL: test_single_report (unittests.tools.test_openreports_parser.TestOpenreportsParser.test_single_report)
2025-11-19T16:31:54.8049654Z uwsgi-1  | ----------------------------------------------------------------------
2025-11-19T16:31:54.8049764Z uwsgi-1  | Traceback (most recent call last):
2025-11-19T16:31:54.8050022Z uwsgi-1  |   File "/app/unittests/tools/test_openreports_parser.py", line 21, in test_single_report
2025-11-19T16:31:54.8050136Z uwsgi-1  |     self.assertEqual(len(findings), 3)
2025-11-19T16:31:54.8050233Z uwsgi-1  |     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
2025-11-19T16:31:54.8050324Z uwsgi-1  | AssertionError: 0 != 3
2025-11-19T16:31:54.8050399Z uwsgi-1  | 
2025-11-19T16:31:54.8050547Z uwsgi-1  | ----------------------------------------------------------------------
2025-11-19T16:31:54.8050636Z uwsgi-1  | Ran 3200 tests in 517.891s
2025-11-19T16:31:54.8050711Z uwsgi-1  | 
2025-11-19T16:31:54.8050812Z uwsgi-1  | FAILED (failures=5, skipped=465)
2025-11-19T16:31:54.8051007Z uwsgi-1  | Preserving test database for alias 'default' ('test_defectdojo')...

@valentijnscholten
Copy link
Member

Thank you for the PR, approved.

Copy link
Contributor

@mtesauro mtesauro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@mtesauro mtesauro merged commit ffc03a9 into DefectDojo:dev Nov 21, 2025
151 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs parser settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR unittests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants