-
Notifications
You must be signed in to change notification settings - Fork 246
feat(compass-collection): CLOUDP-348131: Schema Editor Dropdowns: Add mongo type to faker method mapping CLOUDP-348131 #7425
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
dbeb256
0aaa8f1
73f6b49
b478cd2
85bf859
2ba4849
7228c83
d3baf49
e1d8a5f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| import type { MongoDBFieldType } from '@mongodb-js/compass-generative-ai'; | ||
| import { MockDataGeneratorStep } from './types'; | ||
|
|
||
| export const StepButtonLabelMap = { | ||
|
|
@@ -10,3 +11,72 @@ export const StepButtonLabelMap = { | |
|
|
||
| export const DEFAULT_DOCUMENT_COUNT = 1000; | ||
| export const MAX_DOCUMENT_COUNT = 100000; | ||
|
|
||
| /** | ||
| * Map of MongoDB types to available Faker methods. | ||
| * Not all Faker methods are included here. | ||
| * More can be found in the Faker.js API: https://fakerjs.dev/api/ | ||
| */ | ||
| export const MONGO_TYPE_TO_FAKER_METHODS: Record< | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Gave Augment link to https://v9.fakerjs.dev/api/ and had it come up with some more additions. (If you upgrade to v10 can do the same with the new API link):
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Adding these! |
||
| MongoDBFieldType, | ||
| Array<string> | ||
| > = { | ||
| String: [ | ||
| 'lorem.word', | ||
| 'lorem.words', | ||
| 'lorem.sentence', | ||
| 'lorem.paragraph', | ||
| 'person.firstName', | ||
| 'person.lastName', | ||
| 'person.fullName', | ||
| 'person.jobTitle', | ||
| 'internet.email', | ||
| 'internet.url', | ||
| 'internet.domainName', | ||
| 'internet.userName', | ||
| 'phone.number', | ||
| 'location.city', | ||
| 'location.country', | ||
| 'location.streetAddress', | ||
| 'location.zipCode', | ||
| 'location.state', | ||
| 'company.name', | ||
| 'company.catchPhrase', | ||
| 'color.human', | ||
| 'commerce.productName', | ||
| 'commerce.department', | ||
| 'finance.accountName', | ||
| 'finance.currencyCode', | ||
| 'git.commitSha', | ||
| 'string.uuid', | ||
| 'string.alpha', | ||
| 'string.alphanumeric', | ||
| ], | ||
| Number: [ | ||
| 'number.int', | ||
| 'number.float', | ||
| 'finance.amount', | ||
| 'location.latitude', | ||
| 'location.longitude', | ||
| ], | ||
| Int32: ['number.int', 'finance.amount'], | ||
| Long: ['number.int', 'number.bigInt'], | ||
| Decimal128: ['number.float', 'finance.amount'], | ||
| Boolean: ['datatype.boolean'], | ||
| Date: [ | ||
| 'date.recent', | ||
| 'date.past', | ||
| 'date.future', | ||
| 'date.anytime', | ||
| 'date.birthdate', | ||
| ], | ||
| Timestamp: ['date.recent', 'date.past', 'date.future', 'date.anytime'], | ||
| ObjectId: ['database.mongodbObjectId'], | ||
| Binary: ['string.hexadecimal', 'string.binary'], | ||
| RegExp: ['lorem.word', 'string.alpha'], | ||
| Code: ['lorem.sentence', 'lorem.paragraph', 'git.commitMessage'], | ||
| MinKey: ['number.int'], | ||
| MaxKey: ['number.int'], | ||
| Symbol: ['lorem.word', 'string.symbol'], | ||
| DBRef: ['database.mongodbObjectId'], | ||
| }; | ||
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,152 @@ | ||||||||||
| import { expect } from 'chai'; | ||||||||||
| import React from 'react'; | ||||||||||
| import { | ||||||||||
| screen, | ||||||||||
| render, | ||||||||||
| cleanup, | ||||||||||
| waitFor, | ||||||||||
| userEvent, | ||||||||||
| } from '@mongodb-js/testing-library-compass'; | ||||||||||
| import sinon from 'sinon'; | ||||||||||
| import FakerMappingSelector from './faker-mapping-selector'; | ||||||||||
| import { UNRECOGNIZED_FAKER_METHOD } from '../../modules/collection-tab'; | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [Drive-by suggestion]: This made me realize, due to parallel implementation there's a slight mismatch with |
||||||||||
| import { MONGO_TYPE_TO_FAKER_METHODS } from './constants'; | ||||||||||
| import { MongoDBFieldTypeValues } from '@mongodb-js/compass-generative-ai'; | ||||||||||
|
|
||||||||||
| const mockActiveJsonType = MongoDBFieldTypeValues.String; | ||||||||||
| const mockActiveFakerFunction = 'lorem.word'; | ||||||||||
| const onJsonTypeSelectStub = sinon.stub(); | ||||||||||
| const onFakerFunctionSelectStub = sinon.stub(); | ||||||||||
|
|
||||||||||
| describe('FakerMappingSelector', () => { | ||||||||||
| afterEach(() => { | ||||||||||
| cleanup(); | ||||||||||
| }); | ||||||||||
|
|
||||||||||
| it('should display all MongoDB types in the dropdown', async () => { | ||||||||||
| // Check that all MongoDB types from the constant are present | ||||||||||
| const mongoTypes = Object.keys(MongoDBFieldTypeValues); | ||||||||||
|
|
||||||||||
| render( | ||||||||||
| <FakerMappingSelector | ||||||||||
| activeJsonType={mockActiveJsonType} | ||||||||||
| activeFakerFunction="lorem.word" | ||||||||||
| onJsonTypeSelect={onJsonTypeSelectStub} | ||||||||||
| onFakerFunctionSelect={onFakerFunctionSelectStub} | ||||||||||
| /> | ||||||||||
| ); | ||||||||||
|
|
||||||||||
| const jsonTypeSelect = screen.getByLabelText('JSON Type'); | ||||||||||
| userEvent.click(jsonTypeSelect); | ||||||||||
|
|
||||||||||
| for (const type of mongoTypes) { | ||||||||||
| await waitFor(() => { | ||||||||||
| expect(screen.getByRole('option', { name: type })).to.exist; | ||||||||||
| }); | ||||||||||
| } | ||||||||||
| }); | ||||||||||
|
|
||||||||||
| describe('should display faker methods for each MongoDB type', () => { | ||||||||||
| Object.entries(MONGO_TYPE_TO_FAKER_METHODS).forEach( | ||||||||||
| ([mongoType, methods]) => { | ||||||||||
|
||||||||||
| Object.entries(MONGO_TYPE_TO_FAKER_METHODS).forEach( | |
| ([mongoType, methods]) => { | |
| for (const [mongoType, methods] of Object.entries(MONGO_TYPE_TO_FAKER_METHODS)) { | |
| it(`should display faker methods for ${mongoType}`, () => {``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,9 +8,11 @@ import { | |
| Select, | ||
| spacing, | ||
| } from '@mongodb-js/compass-components'; | ||
| import React from 'react'; | ||
| import React, { useMemo } from 'react'; | ||
| import { UNRECOGNIZED_FAKER_METHOD } from '../../modules/collection-tab'; | ||
| import type { MongoDBFieldType } from '@mongodb-js/compass-generative-ai'; | ||
| import { MongoDBFieldTypeValues } from '@mongodb-js/compass-generative-ai'; | ||
| import { MONGO_TYPE_TO_FAKER_METHODS } from './constants'; | ||
|
|
||
| const fieldMappingSelectorsStyles = css({ | ||
| width: '50%', | ||
|
|
@@ -37,6 +39,17 @@ const FakerMappingSelector = ({ | |
| onJsonTypeSelect, | ||
| onFakerFunctionSelect, | ||
| }: Props) => { | ||
| const fakerMethodOptions = useMemo(() => { | ||
| const methods = | ||
| MONGO_TYPE_TO_FAKER_METHODS[activeJsonType as MongoDBFieldType] || []; | ||
|
||
|
|
||
| if (methods.includes(activeFakerFunction)) { | ||
| return methods; | ||
| } | ||
|
|
||
| return [activeFakerFunction, ...methods]; | ||
| }, [activeJsonType, activeFakerFunction]); | ||
|
|
||
| return ( | ||
| <div className={fieldMappingSelectorsStyles}> | ||
| <Body className={labelStyles}>Mapping</Body> | ||
|
|
@@ -46,8 +59,7 @@ const FakerMappingSelector = ({ | |
| value={activeJsonType} | ||
| onChange={(value) => onJsonTypeSelect(value as MongoDBFieldType)} | ||
| > | ||
| {/* TODO(CLOUDP-344400) : Make the select input editable and render other options depending on the JSON type selected */} | ||
| {[activeJsonType].map((type) => ( | ||
| {Object.values(MongoDBFieldTypeValues).map((type) => ( | ||
| <Option key={type} value={type}> | ||
| {type} | ||
| </Option> | ||
|
|
@@ -59,10 +71,9 @@ const FakerMappingSelector = ({ | |
| value={activeFakerFunction} | ||
| onChange={onFakerFunctionSelect} | ||
| > | ||
| {/* TODO(CLOUDP-344400): Make the select input editable and render other JSON types */} | ||
| {[activeFakerFunction].map((field) => ( | ||
| <Option key={field} value={field}> | ||
| {field} | ||
| {fakerMethodOptions.map((method) => ( | ||
| <Option key={method} value={method}> | ||
| {method} | ||
| </Option> | ||
| ))} | ||
| </Select> | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||
|---|---|---|---|---|
|
|
@@ -14,6 +14,7 @@ import FieldSelector from './schema-field-selector'; | |||
| import FakerMappingSelector from './faker-mapping-selector'; | ||||
| import type { FakerSchema, MockDataGeneratorState } from './types'; | ||||
| import type { MongoDBFieldType } from '@mongodb-js/compass-generative-ai'; | ||||
| import { getDefaultFakerMethod } from './script-generation-utils'; | ||||
|
||||
| import { getDefaultFakerMethod } from './script-generation-utils'; |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similarly to my comment about faker args, can we persist the LLM-suggested faker method so that if the user changes the Mongo Type and then reselects the original type, we default to the LLM-suggested faker method? Like with the args, we can store in state.
Quick AI mock-up just as an example, please review ofc:
// Add this state near the other useState declarations (around line 61)
const [originalLlmData, setOriginalLlmData] = useState<Record<string, {
mongoType: MongoDBFieldType;
fakerMethod: string;
fakerArgs: any[];
}>>({});
// Add this useEffect after the existing state declarations
useEffect(() => {
const originalData = Object.fromEntries(
Object.entries(fakerSchema).map(([field, mapping]) => [
field,
{
mongoType: mapping.mongoType,
fakerMethod: mapping.fakerMethod,
fakerArgs: mapping.fakerArgs
}
])
);
setOriginalLlmData(originalData);
}, [fakerSchema]);
// Update the onJsonTypeSelect function (around line 73)
const onJsonTypeSelect = (newJsonType: MongoDBFieldType) => {
const currentMapping = fakerSchemaFormValues[activeField];
const originalData = originalLlmData[activeField];
if (currentMapping) {
// If switching back to original type, restore original LLM data
const shouldRestoreOriginal = originalData &&
newJsonType === originalData.mongoType &&
originalData.fakerMethod !== UNRECOGNIZED_FAKER_METHOD;
const fakerMethod = shouldRestoreOriginal
? originalData.fakerMethod
: getDefaultFakerMethod(newJsonType);
const fakerArgs = shouldRestoreOriginal
? originalData.fakerArgs
: [];
setFakerSchemaFormValues({
...fakerSchemaFormValues,
[activeField]: {
...currentMapping,
mongoType: newJsonType,
fakerMethod,
fakerArgs,
},
});
resetIsSchemaConfirmed();
}
};
// Update the onFakerFunctionSelect function (around line 91)
const onFakerFunctionSelect = (newFakerFunction: string) => {
const currentMapping = fakerSchemaFormValues[activeField];
const originalData = originalLlmData[activeField];
if (currentMapping) {
// Restore original args if user reselects original LLM method
const fakerArgs = (originalData?.fakerMethod === newFakerFunction)
? originalData.fakerArgs
: currentMapping.fakerArgs;
setFakerSchemaFormValues({
...fakerSchemaFormValues,
[activeField]: {
...currentMapping,
fakerMethod: newFakerFunction,
fakerArgs,
},
});
resetIsSchemaConfirmed();
}
};
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, we can restore the original args if the user reselects an LLM suggested method. Since we will be restoring it back to the original args, we won't be needing state to store the original LLM mappings.
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we store these args so if the user re-selects the LLM-suggested method, they're re-displayed? I think we can store it in the state of faker-schema-editor-screen.tsx with a useEffect dependent on fakerSchema, and then in onFakerFunctionSelect restore original args if user reselects original LLM method
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quick example code mock-up in this comment
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -9,6 +9,7 @@ import { | |||||||||||||||||||||||||||||
| BannerVariant, | ||||||||||||||||||||||||||||||
| Body, | ||||||||||||||||||||||||||||||
| DocumentList, | ||||||||||||||||||||||||||||||
| useDarkMode, | ||||||||||||||||||||||||||||||
| } from '@mongodb-js/compass-components'; | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| import { usePreference } from 'compass-preferences-model/provider'; | ||||||||||||||||||||||||||||||
|
|
@@ -23,11 +24,14 @@ interface RawSchemaConfirmationScreenProps { | |||||||||||||||||||||||||||||
| fakerSchemaGenerationStatus: MockDataGeneratorState['status']; | ||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| const documentContainerStyles = css({ | ||||||||||||||||||||||||||||||
| backgroundColor: palette.gray.light3, | ||||||||||||||||||||||||||||||
| border: `1px solid ${palette.gray.light2}`, | ||||||||||||||||||||||||||||||
| borderRadius: spacing[400], | ||||||||||||||||||||||||||||||
| }); | ||||||||||||||||||||||||||||||
| const getDocumentContainerStyles = (isDarkMode: boolean) => | ||||||||||||||||||||||||||||||
| css({ | ||||||||||||||||||||||||||||||
| backgroundColor: isDarkMode ? palette.gray.dark3 : palette.gray.light3, | ||||||||||||||||||||||||||||||
| border: `1px solid ${ | ||||||||||||||||||||||||||||||
| isDarkMode ? palette.gray.dark2 : palette.gray.light2 | ||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
| isDarkMode ? palette.gray.dark2 : palette.gray.light2 | |
| const documentContainerStyles = css({ | |
| border: `1px solid ${palette.gray.light2}`, | |
| ... | |
| }); | |
| const documentContainerDarkStyles = css({ | |
| border: `1px solid ${palette.gray.dark2}`, | |
| }); | |
| ... | |
| return ( | |
| <div className={cx(documentContainerStyles, isDarkmode && documentContainerDarkStyles)} /> | |
| ); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. This works, but maybe we can just memoize the CSS object? That way, we don't need to have both a light and dark mode documentContainerStyles.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess we can do that, but this would introduce another pattern in codebase. Would be better if we stick to how its already done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok yeah, that's the only doubt I had. I'll stick to the current pattern then to keep it simple and readable!
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is linking to faker v10, while we're currently on v9— maybe we should just take this opportunity to bump to v10. Think we could just update in
package.json:"@faker-js/faker": "^10.0.0", I think should be straightforward, only change I see on first pass would beinternet.userName->internet.usernamein this fileThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The compass repo is set up to use common JS and not ESM, which is what faker v10 requires. I'm wondering if we can tackle this in a separate PR if we want to upgrade to v10 so that we can get this ticket in first.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Compass repo can handle esm in application code just fine, but I think you downgraded this to 9 because that's what the model generates better, at least based on this comment #7295 (comment)
Would be nice if you document this somewhere so that the reason is known
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jcobis What do you think about this? Can you document which version we want in the docs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok sounds good to me, we can stick with v9 for now