refactor(prompts): improve spec generation workflow and task list formatting #30
Conversation
|
Warning Rate limit exceeded@ryderstorm has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 22 minutes and 32 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (4)
WalkthroughReplaces “demo criteria” with “proof artifacts” across prompts, formalizes spec-directory creation and naming, restructures spec-generation steps (questions file process with STOP/WAIT and iterative rounds), and changes task-list and validation outputs to hierarchical, artifact-driven formats and saving conventions. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant User
participant Agent
participant FS as FileSystem
rect rgb(230,245,230)
Note over Agent,FS: Spec generation with questions-file rounds
end
User->>Agent: Request spec generation for feature
Agent->>FS: Create & verify spec directory (./docs/specs/.../<NN>-<title>/)
FS-->>Agent: Confirm directory exists
Agent->>User: Emit Context Assessment & Initial Scope prompts
User-->>Agent: Populate Questions File (round-01.md) — STOP/WAIT
Agent->>User: Iterate clarifying rounds as needed (round-02.md...)
Agent->>Agent: Build spec.md (Units, FRs, Proof Artifacts)
Agent->>FS: Save `<NN>-spec-<feature>.md` in spec directory
FS-->>Agent: Confirm saved
Agent->>User: Present spec for Review/Refine (loop)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
prompts/generate-spec.md (3)
139-179: Questions File Workflow introduces needed structure but needs clarity on file discovery.The iterative rounds workflow is well-designed:
[NN]-questions-[N]-[feature-name].mdwith incrementing[N]for each round. However, the handoff at line 172 ("After the user indicates they have saved their answers, read the file and continue the conversation") assumes the AI will proactively read the file. Consider adding explicit guidance: e.g., "Ask the user to respond 'Answers saved' when ready, then use the Read tool to retrieve the file." This prevents ambiguity about when to proceed.Additionally, line 177 mentions "use context from earlier responses"—clarify whether earlier responses are retained from in-memory conversation history or should be extracted from previous questions files (e.g.,
[NN]-questions-1-[feature-name].md). This is critical for multi-round clarity.
141-141: Minor grammar: Use hyphen for compound modifier.The phrase "multiple choice answers" should be "multiple-choice answers" (line 141) to follow standard English hyphenation rules for compound modifiers preceding a noun.
Apply this diff:
-Ask clarifying questions to gather sufficient detail. **Always provide multiple choice answers** with numbered or lettered options to allow the user to make selections easily. Focus on understanding the "what" and "why" rather than the "how." +Ask clarifying questions to gather sufficient detail. **Always provide multiple-choice answers** with numbered or lettered options to allow the user to make selections easily. Focus on understanding the "what" and "why" rather than the "how."
236-245: Repository Standards section in spec template appropriately reinforces workflow alignment.The expanded guidance in the spec generation template (lines 236–245) now explicitly asks for "Coding standards and style guides," "Architectural patterns," "Testing conventions," and "Commit conventions." This mirrors the context assessment step and ensures generated specs are grounded in repository practices. However, the parenthetical note at line 245 ("If no specific standards are identified, state 'Follow established repository patterns and conventions.'") is a sensible fallback but slightly circular—consider requiring developers to cite at least one repository artifact (e.g., "See CONTRIBUTING.md section X") to enforce rigor.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
prompts/generate-spec.md(8 hunks)prompts/generate-task-list-from-spec.md(2 hunks)
🧰 Additional context used
🪛 LanguageTool
prompts/generate-spec.md
[grammar] ~141-~141: Use a hyphen to join words.
Context: ...icient detail. Always provide multiple choice answers with numbered or letter...
(QB_NEW_EN_HYPHEN)
🔇 Additional comments (7)
prompts/generate-task-list-from-spec.md (2)
137-191: Phase 2 output format is clear and actionable.The hierarchical structure with Demo Criteria → Proof Artifacts → Tasks (TBD) provides a well-organized template. The "TBD" placeholder correctly signals that sub-tasks are deferred to Phase 3. The use of checkboxes ([ ]) and consistent numbering (1.0, 2.0, 3.0) makes the format easy to follow and parse programmatically.
193-266: Phase 3 format maintains consistency while expanding task detail.The parallel structure between Phase 2 and Phase 3 is excellent—only the Tasks section changes from "TBD" to enumerated sub-tasks (1.1, 1.2, 2.1, 2.2, etc.). This maintains cognitive continuity while signaling clear progression. The "Relevant Files" section (lines 198-212) at the top of Phase 3 provides necessary context before task breakdown.
prompts/generate-spec.md (5)
58-64: Process overview clearly signals workflow expansion.The updated sequence (Steps 1–6) makes the progression explicit: Create Directory → Assess Context → Evaluate Scope → Ask Questions → Generate Spec → Review. This is a logical progression that ensures foundational steps (directory creation, context gathering) precede specification work. The numbering is clear and easy to reference.
65-75: Step 1: Create Spec Directory addition ensures consistent project structure.Formalizing directory creation as Step 1 is a solid best practice—it guarantees all downstream files (questions, spec, tasks, proofs) land in a predictable location. The naming convention (
[NN]-spec-[feature-name]/) is clear, and the verification checkpoint ("Confirm the directory exists before proceeding to Step 2") prevents downstream path errors.
77-93: Context Assessment step appropriately balances guidance with optionality.Line 79's clarification—"If working in a pre-existing project, begin by..."—sets realistic expectations. The expanded Repository Standards guidance (lines 85–91) is well-placed and actionable. Line 93's directive ("Use this context to inform scope validation and requirements, not to drive technical decisions") correctly prevents premature design decisions.
267-282: Review and Refinement + Output Requirements sections are well-placed.Step 6 (lines 267–276) provides concrete review prompts to ensure spec completeness. The Output Requirements section (lines 278–282) now includes a concrete example ("For feature 'user authentication'...") that demonstrates the naming and path conventions, making it much easier for users to apply. This is a helpful addition.
309-325: Final Instructions sequence is now complete and actionable.The updated final instructions enumerate the full 9-step workflow (Create Directory → Assessment → Scope → Questions → Spec Generation → Save → Review → Guide → Stop). This is comprehensive and leaves no ambiguity about the expected sequence. Line 323 ("Save file as...") is now explicit (previously implied). The emphasis on "Stop working once user confirms spec is complete" (line 325) appropriately caps the phase.
RobertKelly
left a comment
RobertKelly
left a comment
There was a problem hiding this comment.
Really just had questions on the directory/file structure and wanted to know if the files were committed or transient. Would love to discuss.
|
|
||
| **Directory Structure:** | ||
|
|
||
| - **Path**: `./docs/specs/[NN]-spec-[feature-name]/` where `[NN]` is a zero-padded 2-digit sequence number (e.g., `01`, `02`, `03`) |
There was a problem hiding this comment.
Is this accurate? I would suggest the feature name or story id would be a folder level here rather than sequences stacking up in the specs folder. Also is the word "spec" redundant if it's in the specs directory?
There was a problem hiding this comment.
I may just be missing the need for the leading sequence in the folder path. Would like to discuss.
prompts/generate-spec.md
Outdated
| **Questions File Workflow:** | ||
|
|
||
| If the feature involves existing systems, briefly review the codebase and existing docs to understand: | ||
| 1. **Create Questions File**: Save questions to `[NN]-questions-[N]-[feature-name].md` where `[N]` is the round number (starting at 1, incrementing for each new round). |
There was a problem hiding this comment.
Should the questions also be in the respective specs folder for each feature? I'm just conceptualizing how this lays out so the folder doesn't just grow and grow with contents from feature to feature. (Assuming they get checked in)
|
|
||
| **Format:** Markdown (`.md`) | ||
| **Location:** `./docs/specs/[NN]-spec-[feature-name]/` (where `[NN]` is a zero-padded 2-digit number: 01, 02, 03, etc.) | ||
| **Filename:** `[NN]-validation-[feature-name].md` (e.g., if the Spec is `01-spec-user-authentication.md`, save as `01-validation-user-authentication.md`) |
There was a problem hiding this comment.
I think I'm understanding the sequence ID but may contend that a story/task unique ID is what I imagined. Again, would just like to discuss so I can come along on the journey.
- Add dedicated Step 1 for spec directory creation - Renumber all subsequent steps (1→2, 2→3, etc.) - Add Questions File Workflow for iterative Q&A process - Update all step references throughout document - Fix typo: 'pre-existing'
Update generate-task-list-from-spec prompt to use hierarchical markdown structure for better readability. - Change from flat bullet lists to hierarchical headers (### and ####) - Convert Demo Criteria and Proof Artifacts to bulleted lists - Add TBD placeholder for Tasks section in Phase 2 output format
Remove bullet point about handling clarification requests in the current questions file, as this is already covered by the workflow steps above.
Add 'Saving The Output' section specifying where validation reports should be saved, aligning with other workflow output file locations in the spec directory structure.
ryderstorm
force-pushed
the
feat/improve-spec-and-task-generation
branch
from
37b159e to
faa1992
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
prompts/generate-spec.md (4)
168-179: Clarify the file location for iterative questions files.The Questions File Workflow introduces rounds of questions files (
[NN]-questions-[N]-[feature-name].md), but it's unclear whether these files should be saved in the spec directory created in Step 1 or elsewhere. Given the[NN]prefix (which matches the spec directory numbering), explicitly state that these files belong in./docs/specs/[NN]-spec-[feature-name]/.This addresses the concern raised in past reviews about how files are organized and prevents ambiguity for workflow users.
Apply this diff to clarify the file location:
**Questions File Workflow:** -1. **Create Questions File**: Save questions to `[NN]-questions-[N]-[feature-name].md` where `[N]` is the round number (starting at 1, incrementing for each new round). +1. **Create Questions File**: Save questions to `./docs/specs/[NN]-spec-[feature-name]/[NN]-questions-[N]-[feature-name].md` where `[N]` is the round number (starting at 1, incrementing for each new round). 2. **Point User to File**: Direct the user to the questions file and instruct them to answer the questions directly in the file. 3. **Wait for Confirmation**: After the user indicates they have saved their answers, read the file and continue the conversation. -4. **Follow-Up Rounds**: If answers reveal new questions, create a new questions file with incremented round number (`[NN]-questions-[N+1]-[feature-name].md`) and repeat the process. +4. **Follow-Up Rounds**: If answers reveal new questions, create a new questions file with incremented round number (`./docs/specs/[NN]-spec-[feature-name]/[NN]-questions-[N+1]-[feature-name].md`) and repeat the process.
141-141: Minor grammar: hyphenate compound adjective."Multiple choice" used as a compound adjective should be hyphenated as "multiple-choice."
Apply this diff:
-Ask clarifying questions to gather sufficient detail. **Always provide multiple choice answers** with numbered or lettered options to allow the user to make selections easily. Focus on understanding the "what" and "why" rather than the "how." +Ask clarifying questions to gather sufficient detail. **Always provide multiple-choice answers** with numbered or lettered options to allow the user to make selections easily. Focus on understanding the "what" and "why" rather than the "how."
204-214: Verify spec template section clarity for implementers.The Demoable Units section now uses hierarchical formatting with Purpose, Demo Criteria, and Proof Artifacts subsections. This is well-structured, but verify that junior developers understand the distinction:
- Purpose: Why this slice exists and who it serves (motivation)
- Demo Criteria: Observable behaviors that prove it works (acceptance test mindset)
- Proof Artifacts: Concrete evidence (URLs, CLI output, test names)
The section headers are clear, but consider adding a brief inline comment if users report confusion between "Demo Criteria" (the what) and "Proof Artifacts" (the evidence).
309-325: Reconcile the process overview with final instructions: step count mismatch.The Process Overview (lines 58–63) lists 6 steps ending with "Review and Refine," but the Final Instructions (lines 309–325) list 9 items, adding "Save," "Review and Refine," "Guide User," and "Stop" as separate steps. This inconsistency may confuse users about the true workflow structure.
Ensure both sections align: either consolidate to 6 steps or expand the Process Overview to match the 9-step final breakdown.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
prompts/generate-spec.md(8 hunks)prompts/generate-task-list-from-spec.md(2 hunks)prompts/validate-spec-implementation.md(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- prompts/validate-spec-implementation.md
🚧 Files skipped from review as they are similar to previous changes (1)
- prompts/generate-task-list-from-spec.md
🧰 Additional context used
🪛 LanguageTool
prompts/generate-spec.md
[grammar] ~141-~141: Use a hyphen to join words.
Context: ...icient detail. Always provide multiple choice answers with numbered or letter...
(QB_NEW_EN_HYPHEN)
🔇 Additional comments (1)
prompts/generate-spec.md (1)
278-282: Output path and example are clear and consistent.The specification file location and example correctly reflect the directory structure created in Step 1 and provide a helpful concrete example.
Add a generic example template showing the expected format for clarifying questions with numbered questions, lettered options, and checkboxes. Remove redundant instruction about providing multiple choice answers.
…units Move functional requirements from separate section into each Demoable Unit to make specs more self-contained and easier to read. Each unit now contains all requirements needed for implementation, reducing cross-referencing and making the spec structure more actionable.
…acts Remove separate 'Demo Criteria' sections and consolidate verification into self-documenting proof artifacts. Each proof artifact now describes what it demonstrates (e.g., 'Screenshot: `--help` output demonstrates new command exists'). Changes: - Remove Demo Criteria from spec and task list templates - Update proof artifacts format to include descriptive language - Update all workflow references from demo criteria to proof artifacts - Rename 'Evidence & Output' to 'Verification Result' in validation table - Update all verification checklists to focus on proof artifacts This simplifies the workflow by making proof artifacts the single source of truth for what needs to be demonstrated.
Refactor validation process to consistently focus on proof artifact verification rather than code implementation analysis. This improves cohesion and clarity throughout the validation prompt. Changes: - Update Evaluation Rubric (R1, R5) to focus on proof artifacts - Refactor Step 4 verification to test proof artifacts instead of searching code files - Rename 'Requirement Implementation' to 'Requirement Coverage' - Update Coverage Matrix examples to use proof artifact results - Restructure Validation Issues section with clearer format and table examples - Remove code analysis directives (line numbers, file content checks) - Add commit log analysis to repository compliance checks This aligns validation with its stated goal: verifying proof artifacts demonstrate spec requirements, not analyzing code implementation.
Add explicit STOP AND WAIT instructions to prevent AI from jumping straight from creating clarifying questions to generating the spec without waiting for user responses. Reorganize workflow section for better clarity and add critical enforcement points.
- Rename sections for better clarity (Process Overview → Spec Generation Overview) - Add explicit requirement to report scope assessment results - Make context assessment required instead of optional - Standardize question format with uppercase option letters - Remove redundant Final Instructions section - Improve formatting consistency throughout
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
Why?
This PR improves the spec-driven development workflow by reorganizing workflow steps and improving output formatting. The changes address usability issues where context was not included in scope evaluation or clarifying questions, and task lists were difficult to scan due to flat formatting.
Evidence:
bd83253reorganizes workflow steps to create directory structure firstebea517improves task list readability with hierarchical markdown3fa1931removes redundant instruction for clarity37b159eadds validation report output location specificationWhat Changed?
Refactored prompt files to improve workflow organization and output formatting. The changes ensure consistent file organization from the start and make task lists significantly easier to read and navigate.
Key Changes:
Workflow Reorganization (
generate-spec.md):[NN]-questions-[N]-[feature-name].md)Task List Format Improvements (
generate-task-list-from-spec.md):###and####headersValidation Output Location (
validate-spec-implementation.md):./docs/specs/[NN]-spec-[feature-name]/[NN]-validation-[feature-name].mdDocumentation Cleanup:
Files Modified:
prompts/generate-spec.md(+52 insertions, -20 deletions)prompts/generate-task-list-from-spec.md(+113 insertions, -41 deletions)prompts/validate-spec-implementation.md(+14 insertions)Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.