diff --git a/Changelog.md b/Changelog.md index 20ec6b2511..64faa78244 100644 --- a/Changelog.md +++ b/Changelog.md @@ -1,5 +1,8 @@ # FOSSA CLI Changelog +## 3.14.0 +- Adds `--x-vendetta` flag for vendored dependency identification ([#1607](https://github.com/fossas/fossa-cli/pull/1607)) + ## 3.13.1 - Add a summary of the snippet scan when the `--x-snippet-scan` flag is used ([#1613](https://github.com/fossas/fossa-cli/pull/1613)) - Update snippet scanning documentation ([#1615](https://github.com/fossas/fossa-cli/pull/1615)) diff --git a/docs/features/vendetta.md b/docs/features/vendetta.md new file mode 100644 index 0000000000..e98e3355a8 --- /dev/null +++ b/docs/features/vendetta.md @@ -0,0 +1,106 @@ + +# Vendetta + +Vendetta is the name of FOSSA's vendored dependency identification feature. + +Vendetta hashes files in your first party source code, compares them against +FOSSA's knowledge base, and matches them to common open source components before +finally feeding those matches to a special algorithm that deduces a holistic set +of vendored open source dependencies present in your project. + +Vendetta can be run as part of `fossa analyze`. To enable it, add the +`--x-vendetta` flag when you run `fossa analyze`: + +```sh +fossa analyze --x-vendetta +``` + +## How Vendetta Works + +When `--x-vendetta` is enabled, the CLI: + +1. **Hashes Files**: Creates MD5 hashes of the contents of all relevant files. +2. **Filters Content**: By default, skips directories like `.git/`, and hidden + directories. This includes, from `.fossa.yml`, + `vendoredDependencies.licenseScanPathFilters.exclude`, documented further + below. +5. **Uploads Hashes**: Sends only the hashes to FOSSA's servers. +6. **Receives Matches**: Gets back information about any matching open source + components. +7. **Infers Dependencies**: Feeds the matches to an algorithm that heuristically + identifies the vendored dependencies in your project. + +## Data Sent to FOSSA + +Vendetta sends _only_ the MD5 hashes of your file contents to FOSSA. The raw +contents are never sent to FOSSA. + +## Data Retention + +The MD5 hashes are stored permanently in FOSSA. + +## Directory Filtering + +By default, Vendetta excludes common non-production directories and follows +`.gitignore` patterns: + +- Hidden directories. +- Globs as directed by `.gitignore` files. + +#### Custom Exclude Filtering + +You can customize which files and directories are excluded from Vendetta by +configuring exclude filters in your `.fossa.yml` file. Note that Vendetta scans +currently only support exclude patterns, not `only` patterns. + +For example: +```yaml +version: 3 +vendoredDependencies: + licenseScanPathFilters: + exclude: + - "**/test/**" + - "**/tests/**" + - "**/spec/**" + - "**/node_modules/**" + - "**/dist/**" + - "**/build/**" + - "**/*.test.js" + - "**/*.spec.ts" +``` + +**Important Notes:** + +- Vendetta scanning only use the `exclude` filters from `licenseScanPathFilters` + — `only` filters are ignored for this use-case. +- Path filters use standard glob patterns (e.g., `**/*` for recursive matching, + `*` for single-directory matching). +- The configuration goes in the + `vendoredDependencies.licenseScanPathFilters.exclude` section. +- These exclude patterns are passed directly to the Ficus scanning engine as + `--exclude` arguments. +- Default exclusions (hidden files, `.gitignore` patterns) are applied in + addition to custom excludes. + +## A note on scan times + +The first time you run Vendetta on a codebase, it may take a long time to scan. +For example, scanning [Linux](https://github.com/torvalds/linux) for the first +time may take upwards of 60 minutes. This is because most of the files in your +codebase will have never been checked against FOSSA's knowledge base for open +source components, which can take time. + +Once you scan the first time however, FOSSA will cache the open source component +matches for each MD5 hash Vendetta provides. This means that subsequent scans of +the same project will be drastically faster. For example, scanning the same +revision of Linux twice in a row should result in the second scan taking only +1-2 minutes. + +The time it takes to scan newer versions of your codebase will depend on how +many files in the new version have not been previously scanned. A file has been +previously scanned if the exact same file has ever been scanned by Vendetta. +FOSSA recommends scanning your codebase on a regular basis to keep scan times +low. Additionally, if you intend on running Vendetta as part of your CI +pipeline, it might be best to do a manual run first on a local machine. That +way, future automated scans of your project will be able to benefit from the +initial caching done in the first scan. diff --git a/docs/references/subcommands/analyze.md b/docs/references/subcommands/analyze.md index 52ad90f2fe..8e65a06df9 100644 --- a/docs/references/subcommands/analyze.md +++ b/docs/references/subcommands/analyze.md @@ -158,6 +158,25 @@ Snippet Scanning must also be enabled for your organization, and is only availab For more detail about how Snippet Scanning works, how to use file filtering during Snippet Scanning, what information is sent to FOSSA's servers and a description of the Snippet Scan Summary, see [the Snippet Scanning feature documentation](../../features/snippet-scanning.md). +### Vendored Dependency Scanning with Vendetta + +Vendetta is a feature that identifies the paths of potential open source code +dependencies vendored in your project by comparing file hashes against FOSSA's +knowledge base. This feature helps find dependencies that are included in your +project directly as source. + +#### Enabling Vendetta + +| Name | Description | +|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `--x-vendetta` | Enable vendored dependency scanning during analysis. This experimental feature hashes your source files and checks them against FOSSA's open source component database. | + +#### More detail + +For more detail about how Vendetta works, how to use file filtering during +scanning, or what information is sent to FOSSA's servers, see +[the Vendetta feature documentation](../../features/vendetta.md). + ### Experimental Options _Important: For support and other general information, refer to the [experimental options overview](../experimental/README.md) before using experimental options._ diff --git a/integration-test/Analysis/FicusSpec.hs b/integration-test/Analysis/FicusSpec.hs deleted file mode 100644 index 554b9fcba7..0000000000 --- a/integration-test/Analysis/FicusSpec.hs +++ /dev/null @@ -1,74 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE QuasiQuotes #-} - -module Analysis.FicusSpec (spec) where - -import App.Fossa.Ficus.Analyze (analyzeWithFicus) -import App.Fossa.Ficus.Types (FicusSnippetScanResults (..)) -import App.Types (ProjectRevision (..)) -import Control.Carrier.Diagnostics (runDiagnostics) -import Control.Carrier.Stack (runStack) -import Control.Carrier.StickyLogger (ignoreStickyLogger) -import Control.Timeout (Duration (Seconds)) -import Data.List (isInfixOf) -import Data.String.Conversion (toText) -import Diag.Result (Result (Failure, Success)) -import Effect.Exec (runExecIO) -import Effect.Logger (ignoreLogger) -import Effect.ReadFS (runReadFSIO) -import Fossa.API.Types (ApiKey (..), ApiOpts (..)) -import Path (Dir, Path, Rel, reldir, ()) -import Path.IO qualified as PIO -import System.Environment (lookupEnv) -import Test.Hspec -import Text.URI (mkURI) - -fixtureDir :: Path Rel Dir -fixtureDir = [reldir|integration-test/Analysis/testdata/ficus|] - -spec :: Spec -spec = do - describe "Ficus snippet scanning integration" $ do - it "should run ficus binary successfully" $ do - -- Check for API configuration from environment - maybeApiKey <- lookupEnv "FOSSA_API_KEY" - maybeEndpoint <- lookupEnv "FOSSA_ENDPOINT" - - apiOpts <- case (maybeApiKey, maybeEndpoint) of - (Just keyStr, Just endpointStr) -> do - uri <- case mkURI (toText endpointStr) of - Just validUri -> pure validUri - Nothing -> fail $ "Invalid API endpoint URL: " ++ endpointStr - let opts = ApiOpts (Just uri) (ApiKey (toText keyStr)) (Seconds 60) - pure (Just opts) - _ -> pure Nothing - - currentDir <- PIO.getCurrentDir - let testDataDir = currentDir fixtureDir - revision = ProjectRevision "ficus-integration-test" "testdata-123456" (Just "integration-test") - - -- Check if test data exists - testDataExists <- PIO.doesDirExist testDataDir - testDataExists `shouldBe` True - - result <- runStack . runDiagnostics . ignoreStickyLogger . ignoreLogger . runExecIO . runReadFSIO $ analyzeWithFicus testDataDir apiOpts revision Nothing (Just 10) Nothing - - case result of - Success _warnings analysisResult -> do - case analysisResult of - Just results -> do - ficusSnippetScanResultsAnalysisId results `shouldSatisfy` (> 0) - Nothing -> do - -- No snippet scan results returned - this is acceptable for integration testing - True `shouldBe` True - Failure _warnings errors -> do - let failureMsg = show errors - case apiOpts of - Just _ -> do - -- With API credentials, accept 404/temp-storage errors as valid connectivity tests - if "404" `isInfixOf` failureMsg || "temp-storage" `isInfixOf` failureMsg || "Status(" `isInfixOf` failureMsg - then True `shouldBe` True -- Expected API connectivity issue - else fail ("Ficus integration test failed unexpectedly: " ++ failureMsg) - Nothing -> do - -- Without API credentials, analysis failure is expected - True `shouldBe` True diff --git a/spectrometer.cabal b/spectrometer.cabal index 8c8d9739b0..80c6ebb7d5 100644 --- a/spectrometer.cabal +++ b/spectrometer.cabal @@ -657,6 +657,7 @@ test-suite unit-tests Erlang.Rebar3TreeSpec Extra.ListSpec Extra.TextSpec + Ficus.FicusSpec Fortran.FpmTomlSpec Fossa.API.TypesSpec Go.GlideLockSpec @@ -758,7 +759,6 @@ test-suite integration-tests Analysis.CocoapodsSpec Analysis.ElixirSpec Analysis.ErlangSpec - Analysis.FicusSpec Analysis.FixtureExpectationUtils Analysis.FixtureUtils Analysis.GoSpec diff --git a/src/App/Fossa/Analyze.hs b/src/App/Fossa/Analyze.hs index 16ad47dd32..7b0be7d0db 100644 --- a/src/App/Fossa/Analyze.hs +++ b/src/App/Fossa/Analyze.hs @@ -51,6 +51,7 @@ import App.Fossa.Config.Analyze ( import App.Fossa.Config.Analyze qualified as Config import App.Fossa.Config.Common (DestinationMeta (..), destinationApiOpts, destinationMetadata) import App.Fossa.Ficus.Analyze (analyzeWithFicus) +import App.Fossa.Ficus.Types (FicusAnalysisResults (vendoredDependencyScanResults), FicusStrategy (FicusStrategySnippetScan, FicusStrategyVendetta), FicusVendoredDependencyScanResults (FicusVendoredDependencyScanResults)) import App.Fossa.FirstPartyScan (runFirstPartyScan) import App.Fossa.Lernie.Analyze (analyzeWithLernie) import App.Fossa.Lernie.Types (LernieResults (..)) @@ -103,12 +104,12 @@ import Data.Flag (Flag, fromFlag) import Data.Foldable (traverse_) import Data.Functor (($>)) import Data.List.NonEmpty qualified as NE -import Data.Maybe (fromMaybe, isJust, mapMaybe) +import Data.Maybe (catMaybes, fromMaybe, isJust, mapMaybe, maybeToList) import Data.String.Conversion (decodeUtf8, toText) import Data.Text.Extra (showT) import Data.Traversable (for) import Diag.Diagnostic as DI -import Diag.Result (Result (Success), resultToMaybe) +import Diag.Result (resultToMaybe) import Discovery.Archive qualified as Archive import Discovery.Filters (AllFilters, MavenScopeFilters, applyFilters, filterIsVSIOnly, ignoredPaths, isDefaultNonProductionPath) import Discovery.Projects (withDiscoveredProjects) @@ -302,6 +303,7 @@ analyze cfg = Diag.context "fossa-analyze" $ do allowedTactics = Config.allowedTacticTypes cfg withoutDefaultFilters = Config.withoutDefaultFilters cfg enableSnippetScan = Config.xSnippetScan cfg + enableVendetta = Config.xVendetta cfg manualSrcUnits <- Diag.errorBoundaryIO . diagToDebug $ @@ -340,27 +342,27 @@ analyze cfg = Diag.context "fossa-analyze" $ do if (fromFlag BinaryDiscovery $ Config.binaryDiscoveryEnabled $ Config.vsiOptions cfg) then analyzeDiscoverBinaries basedir filters else pure Nothing + let ficusStrategies = + catMaybes + [ if enableSnippetScan then Just FicusStrategySnippetScan else Nothing + , if enableVendetta then Just FicusStrategyVendetta else Nothing + ] maybeFicusResults <- Diag.errorBoundaryIO . diagToDebug $ - if not enableSnippetScan + if null ficusStrategies || filterIsVSIOnly filters then do - logInfo "Skipping ficus snippet scanning (--x-snippet-scan not set)" pure Nothing else - if filterIsVSIOnly filters - then do - logInfo "Running in VSI only mode, skipping snippet-scan" - pure Nothing - else - Diag.context "snippet-scanning" - . runStickyLogger SevInfo - $ analyzeWithFicus - basedir - maybeApiOpts - revision - (Config.licenseScanPathFilters vendoredDepsOptions) - (orgSnippetScanSourceCodeRetentionDays =<< orgInfo) - (Config.debugDir cfg) + Diag.context "ficus-scanning" + . runStickyLogger SevInfo + $ analyzeWithFicus + basedir + maybeApiOpts + revision + ficusStrategies + (Config.licenseScanPathFilters vendoredDepsOptions) + (orgSnippetScanSourceCodeRetentionDays =<< orgInfo) + (Config.debugDir cfg) let ficusResults = join $ resultToMaybe maybeFicusResults maybeLernieResults <- @@ -378,13 +380,22 @@ analyze cfg = Diag.context "fossa-analyze" $ do vsiResults' :: [SourceUnit] vsiResults' = fromMaybe [] $ join (resultToMaybe vsiResults) + ficusResults' :: [SourceUnit] + ficusResults' = + maybeToList $ + ficusResults + >>= vendoredDependencyScanResults + >>= \(FicusVendoredDependencyScanResults maybeSrcUnit) -> maybeSrcUnit + additionalSourceUnits :: [SourceUnit] - additionalSourceUnits = vsiResults' <> mapMaybe (join . resultToMaybe) [manualSrcUnits, binarySearchResults, dynamicLinkedResults] + additionalSourceUnits = vsiResults' <> ficusResults' <> mapMaybe (join . resultToMaybe) [manualSrcUnits, binarySearchResults, dynamicLinkedResults] traverse_ (Diag.flushLogs SevError SevDebug) [manualSrcUnits, binarySearchResults, dynamicLinkedResults] -- Flush logs using the original Result from VSI. traverse_ (Diag.flushLogs SevError SevDebug) [vsiResults] -- Flush logs from lernie traverse_ (Diag.flushLogs SevError SevDebug) [maybeLernieResults] + -- Flush logs from ficus + traverse_ (Diag.flushLogs SevError SevDebug) [maybeFicusResults] maybeFirstPartyScanResults <- Diag.errorBoundaryIO . diagToDebug $ @@ -450,7 +461,7 @@ analyze cfg = Diag.context "fossa-analyze" $ do $ analyzeForReachability projectScans let reachabilityUnits = onlyFoundUnits reachabilityUnitsResult - let analysisResult = AnalysisScanResult projectScans vsiResults binarySearchResults (Success [] Nothing) manualSrcUnits dynamicLinkedResults maybeLernieResults reachabilityUnitsResult + let analysisResult = AnalysisScanResult projectScans vsiResults binarySearchResults maybeFicusResults manualSrcUnits dynamicLinkedResults maybeLernieResults reachabilityUnitsResult isDebugMode = isJust (Config.debugDir cfg) renderScanSummary isDebugMode maybeEndpointAppVersion analysisResult cfg diff --git a/src/App/Fossa/Analyze/Types.hs b/src/App/Fossa/Analyze/Types.hs index 556f61ef16..afa6d59457 100644 --- a/src/App/Fossa/Analyze/Types.hs +++ b/src/App/Fossa/Analyze/Types.hs @@ -12,7 +12,7 @@ module App.Fossa.Analyze.Types ( import App.Fossa.Analyze.Project (ProjectResult) import App.Fossa.Config.Analyze (ExperimentalAnalyzeConfig) -import App.Fossa.Ficus.Types (FicusSnippetScanResults) +import App.Fossa.Ficus.Types (FicusAnalysisResults) import App.Fossa.Lernie.Types (LernieResults) import App.Fossa.Reachability.Types (SourceUnitReachability (..)) import App.Types (Mode) @@ -81,7 +81,7 @@ data AnalysisScanResult = AnalysisScanResult { analyzersScanResult :: [DiscoveredProjectScan] , vsiScanResult :: Result (Maybe [SourceUnit]) , binaryDepsScanResult :: Result (Maybe SourceUnit) - , ficusResult :: Result (Maybe FicusSnippetScanResults) + , ficusResult :: Result (Maybe FicusAnalysisResults) , fossaDepsScanResult :: Result (Maybe SourceUnit) , dynamicLinkingResult :: Result (Maybe SourceUnit) , lernieResult :: Result (Maybe LernieResults) diff --git a/src/App/Fossa/Analyze/Upload.hs b/src/App/Fossa/Analyze/Upload.hs index bfe8a1897d..9ee714428b 100644 --- a/src/App/Fossa/Analyze/Upload.hs +++ b/src/App/Fossa/Analyze/Upload.hs @@ -9,7 +9,7 @@ module App.Fossa.Analyze.Upload ( import App.Fossa.API.BuildLink (getFossaBuildUrl) import App.Fossa.Config.Analyze (JsonOutput (JsonOutput)) -import App.Fossa.Ficus.Types (FicusSnippetScanResults) +import App.Fossa.Ficus.Types (FicusAnalysisResults (..)) import App.Fossa.Reachability.Types (SourceUnitReachability) import App.Fossa.Reachability.Upload (upload) import App.Types ( @@ -107,7 +107,7 @@ uploadSuccessfulAnalysis :: ProjectRevision -> ScanUnits -> [SourceUnitReachability] -> - Maybe FicusSnippetScanResults -> + Maybe FicusAnalysisResults -> m Locator uploadSuccessfulAnalysis (BaseDir basedir) metadata jsonOutput revision scanUnits reachabilityUnits ficusResults = context "Uploading analysis" $ do @@ -125,7 +125,7 @@ uploadSuccessfulAnalysis (BaseDir basedir) metadata jsonOutput revision scanUnit logInfo ("Using branch: `" <> pretty branchText <> "`") uploadResult <- case scanUnits of - SourceUnitOnly units -> uploadAnalysis revision metadata units ficusResults + SourceUnitOnly units -> uploadAnalysis revision metadata units (snippetScanResults =<< ficusResults) LicenseSourceUnitOnly licenseSourceUnit -> do let mergedUnits = mergeSourceAndLicenseUnits [] licenseSourceUnit runStickyLogger SevInfo . uploadAnalysisWithFirstPartyLicensesToS3AndCore revision metadata mergedUnits ficusResults $ orgFileUpload org @@ -166,12 +166,12 @@ uploadAnalysisWithFirstPartyLicensesToS3AndCore :: ProjectRevision -> ProjectMetadata -> NE.NonEmpty FullSourceUnit -> - Maybe FicusSnippetScanResults -> + Maybe FicusAnalysisResults -> FileUpload -> m UploadResponse uploadAnalysisWithFirstPartyLicensesToS3AndCore revision metadata mergedUnits ficusResults uploadKind = do _ <- uploadAnalysisWithFirstPartyLicensesToS3 revision mergedUnits - uploadAnalysisWithFirstPartyLicenses revision metadata uploadKind ficusResults + uploadAnalysisWithFirstPartyLicenses revision metadata uploadKind (snippetScanResults =<< ficusResults) uploadAnalysisWithFirstPartyLicensesToS3 :: ( Has Diagnostics sig m diff --git a/src/App/Fossa/Config/Analyze.hs b/src/App/Fossa/Config/Analyze.hs index 9d457f9197..a2e6c608bc 100644 --- a/src/App/Fossa/Config/Analyze.hs +++ b/src/App/Fossa/Config/Analyze.hs @@ -240,6 +240,7 @@ data AnalyzeCliOpts = AnalyzeCliOpts , analyzeWithoutDefaultFilters :: Flag WithoutDefaultFilters , analyzeStrictMode :: Flag StrictMode , analyzeSnippetScan :: Bool + , analyzeVendetta :: Bool } deriving (Eq, Ord, Show) @@ -280,6 +281,7 @@ data AnalyzeConfig = AnalyzeConfig , mode :: Mode , xSnippetScan :: Bool , debugDir :: Maybe FilePath + , xVendetta :: Bool } deriving (Eq, Ord, Show, Generic) @@ -352,6 +354,7 @@ cliParser = <*> withoutDefaultFilterParser fossaAnalyzeDefaultFilterDocUrl <*> flagOpt StrictMode (applyFossaStyle <> long "strict" <> stringToHelpDoc "Enforces strict analysis to ensure the most accurate results by rejecting fallbacks.") <*> switch (applyFossaStyle <> long "x-snippet-scan" <> stringToHelpDoc "Experimental flag to enable snippet scanning to identify open source code snippets using fingerprinting.") + <*> switch (applyFossaStyle <> long "x-vendetta" <> stringToHelpDoc "Experimental flag to enable vendored dependency scanning to identify open source components using file hashing.") where fossaDepsFileHelp :: Maybe (Doc AnsiStyle) fossaDepsFileHelp = @@ -568,6 +571,7 @@ mergeStandardOpts maybeDebugDir maybeConfig envvars cliOpts@AnalyzeCliOpts{..} = <*> pure mode <*> pure analyzeSnippetScan <*> pure maybeDebugDir + <*> pure analyzeVendetta collectMavenScopeFilters :: (Has Diagnostics sig m) => diff --git a/src/App/Fossa/Ficus/Analyze.hs b/src/App/Fossa/Ficus/Analyze.hs index f5142da06b..61d975c728 100644 --- a/src/App/Fossa/Ficus/Analyze.hs +++ b/src/App/Fossa/Ficus/Analyze.hs @@ -13,6 +13,7 @@ import App.Fossa.EmbeddedBinary (BinaryPaths, toPath, withFicusBinary) import App.Fossa.Ficus.Types ( FicusAllFlag (..), FicusAnalysisFlag (..), + FicusAnalysisResults (..), FicusConfig (..), FicusDebug (..), FicusError (..), @@ -23,6 +24,9 @@ import App.Fossa.Ficus.Types ( FicusPerStrategyFlag (..), FicusScanStats (..), FicusSnippetScanResults (..), + FicusStrategy (FicusStrategySnippetScan, FicusStrategyVendetta), + FicusVendoredDependency (..), + FicusVendoredDependencyScanResults (..), ) import App.Types (ProjectRevision (..)) import Control.Applicative ((<|>)) @@ -31,6 +35,7 @@ import Control.Concurrent.Async (async, wait) import Control.Effect.Lift (Has, Lift, sendIO) import Control.Monad (when) import Data.Aeson (decode, decodeStrictText) +import Data.Aeson qualified as Aeson import Data.ByteString.Lazy qualified as BL import Data.Conduit ((.|)) import Data.Conduit qualified as Conduit @@ -50,7 +55,7 @@ import Effect.Logger (Logger, logDebug, logInfo) import Fossa.API.Types (ApiKey (..), ApiOpts (..)) import Path (Abs, Dir, Path, toFilePath) import Prettyprinter (pretty) -import Srclib.Types (Locator (..), renderLocator) +import Srclib.Types (Locator (..), SourceUnit (..), SourceUnitBuild (..), SourceUnitDependency (..), renderLocator, textToOriginPath) import System.FilePath (()) import System.IO (Handle, IOMode (WriteMode), hClose, hGetLine, hIsEOF, hPutStrLn, openFile, stderr) import System.Process.Typed ( @@ -67,7 +72,7 @@ import System.Process.Typed ( import Text.Printf (printf) import Text.URI (render) import Text.URI.Builder (PathComponent (PathComponent), TrailingSlash (TrailingSlash), setPath) -import Types (GlobFilter (..), LicenseScanPathFilters (..)) +import Types (GlobFilter (..), GraphBreadth (..), LicenseScanPathFilters (..)) import Prelude newtype CustomLicensePath = CustomLicensePath {unCustomLicensePath :: Text} @@ -92,12 +97,13 @@ analyzeWithFicus :: Path Abs Dir -> Maybe ApiOpts -> ProjectRevision -> + [FicusStrategy] -> Maybe LicenseScanPathFilters -> Maybe Int -> Maybe FilePath -> -- Debug directory (if enabled) - m (Maybe FicusSnippetScanResults) -analyzeWithFicus rootDir apiOpts revision filters snippetScanRetentionDays maybeDebugDir = do - analyzeWithFicusMain rootDir apiOpts revision filters snippetScanRetentionDays maybeDebugDir + m (Maybe FicusAnalysisResults) +analyzeWithFicus rootDir apiOpts revision strategies filters snippetScanRetentionDays maybeDebugDir = do + Just <$> analyzeWithFicusMain rootDir apiOpts revision strategies filters snippetScanRetentionDays maybeDebugDir analyzeWithFicusMain :: ( Has Diagnostics sig m @@ -107,18 +113,20 @@ analyzeWithFicusMain :: Path Abs Dir -> Maybe ApiOpts -> ProjectRevision -> + [FicusStrategy] -> Maybe LicenseScanPathFilters -> Maybe Int -> Maybe FilePath -> -- Debug directory (if enabled) - m (Maybe FicusSnippetScanResults) -analyzeWithFicusMain rootDir apiOpts revision filters snippetScanRetentionDays maybeDebugDir = do + m FicusAnalysisResults +analyzeWithFicusMain rootDir apiOpts revision strategies filters snippetScanRetentionDays maybeDebugDir = do logDebugWithTime "Preparing Ficus analysis configuration..." ficusResults <- runFicus maybeDebugDir ficusConfig logDebugWithTime "runFicus completed, processing results..." - case ficusResults of - Just results -> - logInfo $ pretty (formatFicusScanSummary results) - Nothing -> logInfo "Ficus analysis completed but no fingerprint findings were found" + when (FicusStrategySnippetScan `elem` strategies) $ + case snippetScanResults ficusResults of + Just results -> + logInfo $ pretty (formatFicusScanSummary results) + Nothing -> logInfo "Ficus analysis completed but no fingerprint findings were found" pure ficusResults where ficusConfig = @@ -130,6 +138,7 @@ analyzeWithFicusMain rootDir apiOpts revision filters snippetScanRetentionDays m , ficusConfigRevision = revision , ficusConfigFlags = [All $ FicusAllFlag SkipHiddenFiles, All $ FicusAllFlag Gitignore] , ficusConfigSnippetScanRetentionDays = snippetScanRetentionDays + , ficusConfigStrategies = strategies } findingToSnippetScanResult :: FicusFinding -> Maybe FicusSnippetScanResults @@ -165,6 +174,62 @@ formatFicusScanSummary results = formatProcessingTime :: Double -> Text formatProcessingTime seconds = toText (printf "%.3f" seconds :: String) +findingToVendoredDependency :: FicusFinding -> Maybe FicusVendoredDependency +findingToVendoredDependency (FicusFinding (FicusMessageData strategy payload)) + | Text.toLower strategy == "vendetta" = + decode (BL.fromStrict $ Text.Encoding.encodeUtf8 payload) +findingToVendoredDependency _ = Nothing + +vendoredDepsToSourceUnit :: [FicusVendoredDependency] -> SourceUnit +vendoredDepsToSourceUnit deps = + SourceUnit + { sourceUnitName = "ficus-vendored-dependencies" + , sourceUnitType = "ficus-vendored" + , sourceUnitManifest = "ficus-vendored-dependencies" + , sourceUnitBuild = + Just $ + SourceUnitBuild + { buildArtifact = "default" + , buildSucceeded = True + , buildImports = locators + , buildDependencies = dependencies + } + , sourceUnitGraphBreadth = Complete + , sourceUnitNoticeFiles = [] + , sourceUnitOriginPaths = map (textToOriginPath . ficusVendoredDependencyPath) deps + , sourceUnitLabels = Nothing + , additionalData = Nothing + } + where + locators :: [Locator] + locators = map vendoredDepToLocator deps + + dependencies :: [SourceUnitDependency] + dependencies = map vendoredDepToSourceUnitDependency deps + + vendoredDepToLocator :: FicusVendoredDependency -> Locator + vendoredDepToLocator dep = + Locator + { locatorFetcher = ficusVendoredDependencyEcosystem dep + , locatorProject = ficusVendoredDependencyName dep + , locatorRevision = ficusVendoredDependencyVersion dep + } + + vendoredDepToSourceUnitDependency :: FicusVendoredDependency -> SourceUnitDependency + vendoredDepToSourceUnitDependency dep = + SourceUnitDependency + { sourceDepLocator = vendoredDepToLocator dep + , sourceDepImports = [] + , sourceDepData = + Aeson.object + [ "vendored" + Aeson..= Aeson.object + [ "type" Aeson..= ("directory" :: Text) + , "path" Aeson..= ficusVendoredDependencyPath dep + ] + ] + } + runFicus :: ( Has Diagnostics sig m , Has (Lift IO) sig m @@ -172,7 +237,7 @@ runFicus :: ) => Maybe FilePath -> FicusConfig -> - m (Maybe FicusSnippetScanResults) + m FicusAnalysisResults runFicus maybeDebugDir ficusConfig = do logDebugWithTime "About to extract Ficus binary..." withFicusBinary $ \bin -> do @@ -236,39 +301,56 @@ runFicus maybeDebugDir ficusConfig = do now <- getCurrentTime pure . formatTime defaultTimeLocale "%H:%M:%S.%3q" $ now - streamFicusOutput :: Handle -> Maybe Handle -> IO (Maybe FicusSnippetScanResults) - streamFicusOutput handle maybeFile = - Conduit.runConduit $ - CC.sourceHandle handle - .| CC.decodeUtf8Lenient - .| CC.linesUnbounded - .| CC.mapM - ( \line -> do - -- Tee raw line to file if debug mode - traverse_ (\fileH -> hPutStrLn fileH (toString line)) maybeFile - pure line - ) - .| CCL.mapMaybe decodeStrictText - .| CC.foldM - ( \acc message -> do - -- Log messages as they come, with timestamps - timestamp <- currentTimeStamp - case message of - FicusMessageError err -> do - hPutStrLn stderr $ "[" ++ timestamp ++ "] ERROR " <> toString (displayFicusError err) - pure acc - FicusMessageDebug dbg -> do - hPutStrLn stderr $ "[" ++ timestamp ++ "] DEBUG " <> toString (displayFicusDebug dbg) - pure acc - FicusMessageFinding finding -> do - hPutStrLn stderr $ "[" ++ timestamp ++ "] FINDING " <> toString (displayFicusFinding finding) - let analysisFinding = findingToSnippetScanResult finding - when (isJust acc && isJust analysisFinding) $ - hPutStrLn stderr $ - "[" ++ timestamp ++ "] ERROR " <> "Found multiple ficus analysis responses." - pure $ acc <|> analysisFinding - ) - Nothing + streamFicusOutput :: Handle -> Maybe Handle -> IO FicusAnalysisResults + streamFicusOutput handle maybeFile = do + accumulator <- + Conduit.runConduit $ + CC.sourceHandle handle + .| CC.decodeUtf8Lenient + .| CC.linesUnbounded + .| CC.mapM + ( \line -> do + -- Tee raw line to file if debug mode + traverse_ (\fileH -> hPutStrLn fileH (toString line)) maybeFile + pure line + ) + .| CCL.mapMaybe decodeStrictText + .| CC.foldM + ( \(currentSnippetResults, currentVendoredDeps) message -> do + -- Log messages as they come, with timestamps + timestamp <- currentTimeStamp + case message of + FicusMessageError err -> do + hPutStrLn stderr $ "[" ++ timestamp ++ "] ERROR " <> toString (displayFicusError err) + pure (currentSnippetResults, currentVendoredDeps) + FicusMessageDebug dbg -> do + hPutStrLn stderr $ "[" ++ timestamp ++ "] DEBUG " <> toString (displayFicusDebug dbg) + pure (currentSnippetResults, currentVendoredDeps) + FicusMessageFinding finding -> do + hPutStrLn stderr $ "[" ++ timestamp ++ "] FINDING " <> toString (displayFicusFinding finding) + let analysisFinding = findingToSnippetScanResult finding + let vendoredDep = findingToVendoredDependency finding + when (isJust currentSnippetResults && isJust analysisFinding) $ + hPutStrLn stderr $ + "[" ++ timestamp ++ "] ERROR " <> "Unexpected mutliple snippet scan results" + let newSnippetResults = currentSnippetResults <|> analysisFinding + let newVendoredDeps = case vendoredDep of + Just dep -> dep : currentVendoredDeps + Nothing -> currentVendoredDeps + pure (newSnippetResults, newVendoredDeps) + ) + (Nothing, []) + + let (snippetResults, vendoredDeps) = accumulator + let vendoredResults = case vendoredDeps of + [] -> Nothing + deps -> Just $ FicusVendoredDependencyScanResults (Just $ vendoredDepsToSourceUnit deps) + + pure $ + FicusAnalysisResults + { snippetScanResults = snippetResults + , vendoredDependencyScanResults = vendoredResults + } consumeStderr :: Handle -> Maybe Handle -> IO [Text] consumeStderr handle maybeFile = do @@ -322,11 +404,15 @@ ficusCommand ficusConfig bin = do pure cmd where snippetScanRetentionDays = ficusConfigSnippetScanRetentionDays ficusConfig - configArgs endpoint = ["analyze", "--secret", secret, "--endpoint", endpoint, "--locator", locator, "--set", "all:skip-hidden-files", "--set", "all:gitignore", "--exclude", ".git", "--exclude", ".git/**"] ++ configExcludes ++ maybe [] (\days -> ["--snippet-scan-retention-days", toText days]) snippetScanRetentionDays ++ [targetDir] + configArgs endpoint = ["analyze", "--secret", secret, "--endpoint", endpoint, "--locator", locator, "--set", "all:skip-hidden-files", "--set", "all:gitignore", "--exclude", ".git", "--exclude", ".git/**"] ++ configExcludes ++ configStrategies ++ maybe [] (\days -> ["--snippet-scan-retention-days", toText days]) snippetScanRetentionDays ++ [targetDir] targetDir = toText $ toFilePath $ ficusConfigRootDir ficusConfig secret = maybe "" (toText . unApiKey) $ ficusConfigSecret ficusConfig locator = renderLocator $ Locator "custom" (projectName $ ficusConfigRevision ficusConfig) (Just $ projectRevision $ ficusConfigRevision ficusConfig) configExcludes = concatMap (\path -> ["--exclude", unGlobFilter path]) $ ficusConfigExclude ficusConfig + configStrategies = concatMap (\strategy -> ["--strategy", strategyToArg strategy]) $ ficusConfigStrategies ficusConfig + strategyToArg = \case + FicusStrategySnippetScan -> "snippet-scanning" + FicusStrategyVendetta -> "vendetta" maskApiKeyInCommand :: Text -> Text maskApiKeyInCommand cmdText = diff --git a/src/App/Fossa/Ficus/Types.hs b/src/App/Fossa/Ficus/Types.hs index 1db7581696..136bad11d2 100644 --- a/src/App/Fossa/Ficus/Types.hs +++ b/src/App/Fossa/Ficus/Types.hs @@ -7,6 +7,7 @@ module App.Fossa.Ficus.Types ( FicusDebug (..), FicusError (..), FicusAnalysisFlag (..), + FicusStrategy (..), FicusAllFlag (..), FicusWalkFlag (..), FicusNoopFlag (..), @@ -14,19 +15,47 @@ module App.Fossa.Ficus.Types ( FicusSnippetScanFlag, FicusSnippetScanResults (..), FicusScanStats (..), + FicusVendettaFlag, FicusPerStrategyFlag (..), + FicusAnalysisResults (..), + FicusVendoredDependency (..), + FicusVendoredDependencyScanResults (..), ) where import App.Types (ProjectRevision) import Data.Aeson (FromJSON (parseJSON), Value (Object), withObject, withText) -import Data.Aeson.Types (Parser, (.:)) +import Data.Aeson.Types (Parser, (.:), (.:?)) import Data.Text (Text) import Fossa.API.Types import GHC.Generics (Generic) import Path (Abs, Dir, Path) +import Srclib.Types (SourceUnit) import Text.URI import Types (GlobFilter) +data FicusAnalysisResults = FicusAnalysisResults + { snippetScanResults :: Maybe FicusSnippetScanResults + , vendoredDependencyScanResults :: Maybe FicusVendoredDependencyScanResults + } + +newtype FicusVendoredDependencyScanResults = FicusVendoredDependencyScanResults (Maybe SourceUnit) + +data FicusVendoredDependency = FicusVendoredDependency + { ficusVendoredDependencyName :: Text + , ficusVendoredDependencyEcosystem :: Text + , ficusVendoredDependencyVersion :: Maybe Text + , ficusVendoredDependencyPath :: Text + } + deriving (Eq, Ord, Show, Generic) + +instance FromJSON FicusVendoredDependency where + parseJSON = withObject "FicusVendoredDependency" $ \obj -> + FicusVendoredDependency + <$> obj .: "name" + <*> obj .: "ecosystem" + <*> obj .:? "version" + <*> obj .: "path" + data FicusSnippetScanResults = FicusSnippetScanResults { ficusSnippetScanResultsAnalysisId :: Int , ficusSnippetScanResultsBucketId :: Int @@ -177,9 +206,15 @@ data FicusConfig = FicusConfig , ficusConfigRevision :: ProjectRevision -- TODO: get this from `projectRevision AnalyzeConfig` , ficusConfigFlags :: [FicusPerStrategyFlag] , ficusConfigSnippetScanRetentionDays :: Maybe Int + , ficusConfigStrategies :: [FicusStrategy] } deriving (Show, Eq, Generic) +data FicusStrategy + = FicusStrategySnippetScan + | FicusStrategyVendetta + deriving (Show, Eq, Generic) + -- A flag for ficus paired with a proper strategy or pseudo-strategy. -- @Walk@ and @All@ are pseudo-strategies which accept similar flags, -- but expand into a subset of strategies in ficus. @@ -189,6 +224,7 @@ data FicusPerStrategyFlag | SnippetScan FicusSnippetScanFlag | Noop FicusNoopFlag | Hash FicusHashFlag + | Vendetta FicusVendettaFlag deriving (Show, Eq, Generic) data FicusAnalysisFlag @@ -206,6 +242,11 @@ newtype FicusNoopFlag = FicusNoopFlag FicusAnalysisFlag deriving (Show, Eq) newtype FicusHashFlag = FicusHashFlag FicusAnalysisFlag deriving (Show, Eq) data FicusSnippetScanFlag - = CommonFlag FicusAnalysisFlag - | BatchLen Int + = SnippetScanCommonFlag FicusAnalysisFlag + | SnippetScanBatchLen Int + deriving (Show, Eq) + +data FicusVendettaFlag + = VendettaCommonFlag FicusAnalysisFlag + | VendettaBatchLen Int deriving (Show, Eq) diff --git a/src/App/Fossa/ManualDeps.hs b/src/App/Fossa/ManualDeps.hs index 00a06fcbaf..adf500ffcf 100644 --- a/src/App/Fossa/ManualDeps.hs +++ b/src/App/Fossa/ManualDeps.hs @@ -358,7 +358,7 @@ mkLinuxPackage :: Text -> Text -> Text -> Text mkLinuxPackage depName os osVersion = depName <> "#" <> os <> "#" <> osVersion addEmptyDep :: Locator -> SourceUnitDependency -addEmptyDep loc = SourceUnitDependency loc [] +addEmptyDep loc = SourceUnitDependency loc [] Data.Aeson.Null toAdditionalData :: Maybe (NE.NonEmpty CustomDependency) -> Maybe (NE.NonEmpty RemoteDependency) -> Maybe AdditionalDepData toAdditionalData customDeps remoteDeps = diff --git a/src/Srclib/Converter.hs b/src/Srclib/Converter.hs index 9270476a51..f1f0d083b1 100644 --- a/src/Srclib/Converter.hs +++ b/src/Srclib/Converter.hs @@ -15,6 +15,7 @@ import Prelude import Algebra.Graph.AdjacencyMap qualified as AM import App.Fossa.Analyze.Project (ProjectResult (..)) import Control.Applicative ((<|>)) +import Data.Aeson qualified as Aeson import Data.Set qualified as Set import Data.String.Conversion (toText) import Data.Text (Text) @@ -35,6 +36,7 @@ import Graphing qualified import Path (toFilePath) import Srclib.Types ( Locator (..), + LocatorWithMetadata (..), OriginPath, SourceUnit (..), SourceUnitBuild ( @@ -91,17 +93,21 @@ toSourceUnit leaveUnfiltered path dependencies projectType graphBreadth originPa locatorAdjacent :: AM.AdjacencyMap Locator locatorAdjacent = Graphing.toAdjacencyMap locatorGraph + locatorsWithMetadata :: [LocatorWithMetadata] + locatorsWithMetadata = map (\loc -> LocatorWithMetadata{locatorWithMetadataLocator = loc, locatorWithMetadataData = Aeson.Null}) (AM.vertexList locatorAdjacent) + deps :: [SourceUnitDependency] - deps = map (mkSourceUnitDependency locatorAdjacent) (AM.vertexList locatorAdjacent) + deps = map (mkSourceUnitDependency locatorAdjacent) locatorsWithMetadata imports :: [Locator] imports = Graphing.directList locatorGraph -mkSourceUnitDependency :: AM.AdjacencyMap Locator -> Locator -> SourceUnitDependency -mkSourceUnitDependency gr locator = +mkSourceUnitDependency :: AM.AdjacencyMap Locator -> LocatorWithMetadata -> SourceUnitDependency +mkSourceUnitDependency gr (LocatorWithMetadata{..}) = SourceUnitDependency - { sourceDepLocator = locator - , sourceDepImports = Set.toList $ AM.postSet locator gr + { sourceDepLocator = locatorWithMetadataLocator + , sourceDepImports = Set.toList $ AM.postSet locatorWithMetadataLocator gr + , sourceDepData = locatorWithMetadataData } shouldPublishDep :: Dependency -> Bool diff --git a/src/Srclib/Types.hs b/src/Srclib/Types.hs index 308996c234..bd37a74c72 100644 --- a/src/Srclib/Types.hs +++ b/src/Srclib/Types.hs @@ -9,6 +9,7 @@ module Srclib.Types ( AdditionalDepData (..), SourceUserDefDep (..), SourceRemoteDep (..), + LocatorWithMetadata (..), Locator (..), LicenseSourceUnit (..), LicenseScanType (..), @@ -409,7 +410,7 @@ data SourceUnitBuild = SourceUnitBuild data SourceUnitDependency = SourceUnitDependency { sourceDepLocator :: Locator , sourceDepImports :: [Locator] -- omitempty - -- , sourceDepData :: Aeson.Value + , sourceDepData :: Data.Aeson.Value } deriving (Eq, Ord, Show) @@ -476,6 +477,12 @@ parseProvidedPackageLabelScope "org" = ProvidedPackageLabelScopeOrg parseProvidedPackageLabelScope "project" = ProvidedPackageLabelScopeProject parseProvidedPackageLabelScope _ = ProvidedPackageLabelScopeRevision +data LocatorWithMetadata = LocatorWithMetadata + { locatorWithMetadataLocator :: Locator + , locatorWithMetadataData :: Data.Aeson.Value + } + deriving (Eq, Ord, Show) + data Locator = Locator { locatorFetcher :: Text , locatorProject :: Text @@ -551,6 +558,7 @@ instance ToJSON SourceUnitDependency where object [ "locator" .= sourceDepLocator , "imports" .= sourceDepImports + , "metadata" .= sourceDepData ] instance FromJSON SourceUnitDependency where @@ -558,6 +566,7 @@ instance FromJSON SourceUnitDependency where SourceUnitDependency <$> obj .: "locator" <*> obj .: "imports" + <*> obj .: "metadata" instance ToJSON AdditionalDepData where toJSON AdditionalDepData{..} = diff --git a/test/App/Fossa/Container/AnalyzeNativeSpec.hs b/test/App/Fossa/Container/AnalyzeNativeSpec.hs index 073f360566..f42ac1597f 100644 --- a/test/App/Fossa/Container/AnalyzeNativeSpec.hs +++ b/test/App/Fossa/Container/AnalyzeNativeSpec.hs @@ -15,6 +15,7 @@ import Control.Carrier.Telemetry (IgnoreTelemetryC, withoutTelemetry) import Control.Effect.DockerEngineApi ( DockerEngineApiF (GetImageSize, IsDockerEngineAccessible), ) +import Data.Aeson qualified as Aeson import Data.Flag (toFlag') import Data.Map qualified as Map import Data.Maybe (mapMaybe) @@ -241,6 +242,7 @@ jarsInContainerSpec = describe "Jars in Containers" $ do [ SourceUnitDependency { sourceDepLocator = depLocator , sourceDepImports = [] + , sourceDepData = Aeson.Null } ] } diff --git a/test/Ficus/FicusSpec.hs b/test/Ficus/FicusSpec.hs new file mode 100644 index 0000000000..ba340e120b --- /dev/null +++ b/test/Ficus/FicusSpec.hs @@ -0,0 +1,70 @@ +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE QuasiQuotes #-} + +module Ficus.FicusSpec (spec) where + +import App.Fossa.Ficus.Analyze (analyzeWithFicus) +import App.Fossa.Ficus.Types (FicusAnalysisResults (..), FicusSnippetScanResults (..), FicusStrategy (FicusStrategySnippetScan, FicusStrategyVendetta), FicusVendoredDependencyScanResults (FicusVendoredDependencyScanResults)) +import App.Types (ProjectRevision (..)) +import Control.Effect.Lift (sendIO) +import Control.Timeout (Duration (Seconds)) +import Data.String.Conversion (toText) +import Fossa.API.Types (ApiKey (..), ApiOpts (..)) +import Path (Dir, Path, Rel, reldir, ()) +import Path.IO qualified as PIO +import Srclib.Types (SourceUnit (sourceUnitName)) +import System.Environment (lookupEnv) +import Test.Effect (expectationFailure', it', shouldBe', shouldSatisfy') +import Test.Hspec +import Text.URI (mkURI) + +fixtureDir :: Path Rel Dir +fixtureDir = [reldir|test/Ficus/testdata|] + +spec :: Spec +spec = do + describe "Ficus snippet scanning integration" $ do + it' "should run ficus binary successfully" $ do + -- Check for API configuration from environment + maybeApiKey <- sendIO $ lookupEnv "FOSSA_API_KEY" + maybeEndpoint <- sendIO $ lookupEnv "FOSSA_ENDPOINT" + + apiOpts <- case (maybeApiKey, maybeEndpoint) of + (Just keyStr, Just endpointStr) -> + case mkURI (toText endpointStr) of + Just uri -> do + let opts = ApiOpts (Just uri) (ApiKey (toText keyStr)) (Seconds 60) + pure (Just opts) + Nothing -> do + expectationFailure' $ "Invalid API endpoint URL: " ++ endpointStr + pure Nothing + _ -> pure Nothing + + currentDir <- sendIO PIO.getCurrentDir + let testDataDir = currentDir fixtureDir + revision = ProjectRevision "ficus-integration-test" "testdata-123456" (Just "integration-test") + + -- Check if test data exists + testDataExists <- sendIO $ PIO.doesDirExist testDataDir + testDataExists `shouldBe'` True + + let strategies = [FicusStrategySnippetScan, FicusStrategyVendetta] + + result <- analyzeWithFicus testDataDir apiOpts revision strategies Nothing (Just 10) Nothing + + case result of + Just results -> do + case snippetScanResults results of + Just snippetResults -> do + ficusSnippetScanResultsAnalysisId snippetResults `shouldSatisfy'` (> 0) + _ -> do + -- No snippet scan results returned - this is acceptable for integration testing + True `shouldBe'` True + + case vendoredDependencyScanResults results of + Just (FicusVendoredDependencyScanResults (Just srcUnit)) -> do + sourceUnitName srcUnit `shouldBe'` "ficus-vendored-dependencies" + _ -> do + -- No vendetta results returned - this is acceptable for integration testing + True `shouldBe'` True + _ -> expectationFailure' "Ficus analysis returned no results unexpectedly." diff --git a/integration-test/Analysis/testdata/ficus/README.md b/test/Ficus/testdata/README.md similarity index 100% rename from integration-test/Analysis/testdata/ficus/README.md rename to test/Ficus/testdata/README.md diff --git a/integration-test/Analysis/testdata/ficus/helper.c b/test/Ficus/testdata/helper.c similarity index 100% rename from integration-test/Analysis/testdata/ficus/helper.c rename to test/Ficus/testdata/helper.c diff --git a/integration-test/Analysis/testdata/ficus/helper.h b/test/Ficus/testdata/helper.h similarity index 100% rename from integration-test/Analysis/testdata/ficus/helper.h rename to test/Ficus/testdata/helper.h diff --git a/integration-test/Analysis/testdata/ficus/main.c b/test/Ficus/testdata/main.c similarity index 100% rename from integration-test/Analysis/testdata/ficus/main.c rename to test/Ficus/testdata/main.c diff --git a/test/Test/Fixtures.hs b/test/Test/Fixtures.hs index dc3ec143b8..00bcb5fecb 100644 --- a/test/Test/Fixtures.hs +++ b/test/Test/Fixtures.hs @@ -75,6 +75,7 @@ import App.Types (Mode (..), OverrideDynamicAnalysisBinary (..)) import App.Types qualified as App import Control.Effect.FossaApiClient qualified as App import Control.Timeout (Duration (MilliSeconds)) +import Data.Aeson qualified as Aeson import Data.ByteString.Lazy qualified as LB import Data.Flag (toFlag) import Data.List.NonEmpty (NonEmpty) @@ -337,12 +338,13 @@ sourceUnitBuildMaven = [ ipAddr , spotBugs ] - [ SourceUnitDependency logger [] - , SourceUnitDependency ipAddr [] + [ SourceUnitDependency logger [] Aeson.Null + , SourceUnitDependency ipAddr [] Aeson.Null , SourceUnitDependency spotBugs [ logger ] + Aeson.Null ] where ipAddr :: Locator @@ -387,6 +389,7 @@ vsiSourceUnit = , locatorRevision = Just "1.2.3" } , sourceDepImports = [] + , sourceDepData = Aeson.Null } ] } @@ -675,6 +678,7 @@ standardAnalyzeConfig = , ANZ.mode = NonStrict , ANZ.xSnippetScan = False , ANZ.debugDir = Nothing + , ANZ.xVendetta = False } sampleJarParsedContent :: Text