-
Notifications
You must be signed in to change notification settings - Fork 844
feat(inspect): Delete model training job #3134
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: feature/geti-inspect
Are you sure you want to change the base?
feat(inspect): Delete model training job #3134
Conversation
Mainly based on the UI files and architecture from [Geti Tune](https://github.com/open-edge-platform/training_extensions/tree/develop/ui)
* initial backend commit Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * app -> src Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * Remove empty file Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * move code Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * fix style backend Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * rename media endpoint Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> --------- Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com>
…#2942) add unit tests for endpoints Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com>
These won't be needed yet
…tform#2948) * Use src folder inside run.sh * Set openapi_url path * Update to react 19 * Specify bash language in readme
…edge-platform#2945) * add training + inference endpoint Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * remove model api Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * cleanup code Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * update async execution Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * improve training worker loop and predict endpoint Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * fix style Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * fix style to use python3.10 generics Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add tests for services Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * style fix Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * style fix Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * style fix Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * style fix Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * style fix Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> --------- Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com>
…edge-platform#2961) * Add github actions for ui and server of geti inspect * Exclude UI from pre-commit prettier configuration The UI uses a different prettier configuration that does not seem to be picked up by pre-commit. * Add newline to .prettierignore * Apply prettier to `geti-inspect.yaml` * Generate OpenAPI spec before running UI checks * Checkout with lfs * Fix lint issues by removing wip components * Try installing git lfs in the playwright docker image * Fix unused noqa
…edge-platform#2963) * Add OpenAPI route * Remove MSW browser worker * Rename infernece to inspect * Update navbar title * Rename infernece to inspect
* rename models/ to pydantic_models/ Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * switch to use async session context manager Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * fix unit tests Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add pipeline endpoints Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * fix frame aquisition worker and rename pipiline endpoints Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add sources and sinks endpoints Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * stream loading working Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add webrtc endpoints Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * fix workers: stream loading + inference + dispatcher Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * fix sinks Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * style Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add unit tests and address comments Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add tests Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * fix example schema Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add todo Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> --------- Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com>
…2970) * chore: Add path alias to icons * feat: Add sidebar with dataset, models and stats
* feat: Display placeholders for images * refactor: Fix scrollbar * chore: Remove learn more
* chore(inspect): Update UI scripts * chore: Update github actions * chore: Update port to 8000
…en-edge-platform#2978) chore(inspect): Configure project(s) mocks for playwright
* chore: Add project route * feat: Add project management * revert ui lock change * chore: Remove not needed code for project management
…ts from the UI (open-edge-platform#2980) chore(inspect): Update allowed origins
… training progress (open-edge-platform#2984) * feat: Allow user to upload images * feat: List uploaded images * chore: Extract components to separate files and add ready to train and training progress * chore: comment thumbnail url generation
chore(inspect): Update openapi page title
…en-edge-platform#2989) * chore(inspect): Renamed app to application * chore(inspect): Rename app to application in github actions
…of email (open-edge-platform#2990) refactor: Update photo placeholder to use indicator instead of email
…dge-platform#2991) Improve error and suspense handling in router By moving all of the routes into a single root route we can make sure that all routes are rendered inside of an layout that has a suspense and error boundary.
Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com>
…open-edge-platform#2992) refactor: Improvements to the jobs management and training
…platform#2994) chore(inspect): Add more models
🐛 fix(inspect): Fix unable to start train job
chore(inspect): Update uv.lock
* add trainable models endpoint * fix Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add test Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add copyright Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> --------- Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com>
…rm#3004) * add thumbnails endpoint Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add tests Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * generate thumbnails as background task Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * update docstring Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> --------- Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com>
…latform#3094) * feat(inspect): implement device selection in training model dialog Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Update application/ui/src/features/inspect/train-model/train-model-device-picker.component.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed tests Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed comments Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Added GitHub link Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed comments Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Update application/ui/src/features/inspect/toolbar/sources/edit-source/edit-source.test.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Added hover effects Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed linter Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Reworked effect hook Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> --------- Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…#3114) * allow stream passthrough Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * fix test Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * update enable pipeline url Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com> * Update application/backend/src/api/endpoints/pipeline_endpoints.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> * fix style Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * improve flow Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * Update application/backend/src/api/endpoints/source_endpoints.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> * Update application/backend/src/api/endpoints/sink_endpoints.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> * fix Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add log error Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * Update application/backend/src/workers/inference.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> * optimize inference with daemon Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * update name Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> --------- Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> Co-authored-by: Colorado, Camilo <camilo.colorado@intel.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…Tune (open-edge-platform#3118) * update metrics Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * fix tests Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * Update application/backend/src/workers/inference.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> * remove comment Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * Update application/backend/src/workers/inference.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> * Update application/backend/src/workers/inference.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> * Update application/backend/src/services/pipeline_metrics_service.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> * fix style Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * Update application/backend/src/services/pipeline_service.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> * Update application/backend/src/workers/inference.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> --------- Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> Signed-off-by: Max Xiang <maxx.rift@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* render enable project screen Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com> * disable previous project Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com> * show sink-source empty message Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com> * integrate active-pipeline api Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com> --------- Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com>
* add stop pipeline endpoint Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * add missing file Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * fix inference bug Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> * apply suggestions Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com> --------- Signed-off-by: Ma, Xiangxiang <xiangxiang.ma@intel.com>
* feat(model): add model size attribute and related database schema updates Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed linter Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed linter Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed comments Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> --------- Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com>
* feat(training): implement cancellation handling for training jobs Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Update application/backend/src/services/training_service.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Update application/backend/src/services/training_service.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Update application/backend/tests/unit/services/test_training_service.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Update application/backend/src/services/training_service.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed linter Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Update application/backend/src/services/training_service.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed comments Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed comments Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed merge Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> --------- Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* update api definitions Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com> * add unit tests Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com> * switch component calls :stop pipeline Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com> * source sink message Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com> * update unit test description Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com> --------- Signed-off-by: Colorado, Camilo <camilo.colorado@intel.com>
* feat(inspect): add endpoint to delete models and their artifacts Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed linter Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Update application/backend/src/services/model_service.py Co-authored-by: Max Xiang <maxx.rift@gmail.com> Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed comments Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed linter Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed comments Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed tests Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Update application/backend/src/services/training_service.py Co-authored-by: Max Xiang <maxx.rift@gmail.com> Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed tests Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> * Fixed tests Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> --------- Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com> Co-authored-by: Max Xiang <maxx.rift@gmail.com>
Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces the inspect feature for the anomalib application, implementing a comprehensive real-time inference pipeline with WebRTC streaming, multiple source/sink configurations, and a complete UI for project management and model training.
Key changes:
- Complete UI implementation for the inspect feature with pipeline configuration, dataset management, and model training capabilities
- Backend services for training, inference, and pipeline management with multiprocessing workers
- WebRTC integration for real-time video streaming and inference
- Support for multiple source types (webcam, IP camera, video files) and sink types (folder, MQTT, ROS, webhook)
Reviewed Changes
Copilot reviewed 1 out of 1 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| application/ui/src/features/inspect/toolbar/sources/add-source/* | Adds source connection functionality with form validation and pipeline integration |
| application/ui/src/features/inspect/toolbar/sinks/* | Implements comprehensive sink configuration UI with support for multiple sink types |
| application/ui/src/features/inspect/toolbar/pipeline-switch/* | Pipeline control switch with WebRTC connection status and test coverage |
| application/ui/src/features/inspect/stream/* | WebRTC stream rendering and container components with connection management |
| application/ui/src/features/inspect/models/* | Model management UI with training status and inference device selection |
| application/backend/src/services/* | Backend services for training, inference, pipeline, and model management |
| application/backend/tests/unit/* | Comprehensive test coverage for services, workers, and endpoints |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com>
Signed-off-by: Dmitry Kalinin <dmitry.kalinin@intel.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
4567ce7 to
ea4d226
Compare
📝 Description
Added deletion of the training job related to the model
✨ Changes
Select what type of change your PR is:
✅ Checklist
Before you submit your pull request, please make sure you have completed the following steps:
For more information about code review checklists, see the Code Review Checklist.