Skip to content

projects: update tools to support pagination, improve prompts #1352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

projects: update tools to support pagination, improve prompts #1352

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c348bb0
pagination, prompt updates
tmelliottjr Nov 5, 2025
a4fa908
update pagination rules
tmelliottjr Nov 5, 2025
68abde2
move query rules
tmelliottjr Nov 5, 2025
5cb0758
resolves PR conflicts with main
stephenotalora Nov 7, 2025
584d2e0
reduce description size/redundancy
tmelliottjr Nov 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -830,24 +830,30 @@ Options are:
- `project_number`: The project's number. (number, required)

- **list_project_fields** - List project fields
- `after`: Forward pagination cursor from previous pageInfo.nextCursor. (string, optional)
- `before`: Backward pagination cursor from previous pageInfo.prevCursor (rare). (string, optional)
- `owner`: If owner_type == user it is the handle for the GitHub user account. If owner_type == org it is the name of the organization. The name is not case sensitive. (string, required)
- `owner_type`: Owner type (string, required)
- `per_page`: Number of results per page (max 100, default: 30) (number, optional)
- `per_page`: Results per page (max 50) (number, optional)
- `project_number`: The project's number. (number, required)

- **list_project_items** - List project items
- `fields`: Specific list of field IDs to include in the response (e.g. ["102589", "985201", "169875"]). If not provided, only the title field is included. (string[], optional)
- `after`: Forward pagination cursor from previous pageInfo.nextCursor. (string, optional)
- `before`: Backward pagination cursor from previous pageInfo.prevCursor (rare). (string, optional)
- `fields`: Field IDs to include (e.g. ["102589", "985201"]). CRITICAL: Always provide to get field values. Without this, only titles returned. Get IDs from list_project_fields first. (string[], optional)
- `owner`: If owner_type == user it is the handle for the GitHub user account. If owner_type == org it is the name of the organization. The name is not case sensitive. (string, required)
- `owner_type`: Owner type (string, required)
- `per_page`: Number of results per page (max 100, default: 30) (number, optional)
- `per_page`: Results per page (max 50) (number, optional)
- `project_number`: The project's number. (number, required)
- `query`: Search query to filter items (string, optional)
- `query`: Query string for advanced filtering of project items using GitHub's project filtering syntax. (string, optional)

- **list_projects** - List projects
- `after`: Forward pagination cursor from previous pageInfo.nextCursor. (string, optional)
- `before`: Backward pagination cursor from previous pageInfo.prevCursor (rare). (string, optional)
- `owner`: If owner_type == user it is the handle for the GitHub user account. If owner_type == org it is the name of the organization. The name is not case sensitive. (string, required)
- `owner_type`: Owner type (string, required)
- `per_page`: Number of results per page (max 100, default: 30) (number, optional)
- `query`: Filter projects by a search query (matches title and description) (string, optional)
- `per_page`: Results per page (max 50) (number, optional)
- `query`: Filter projects by title text and open/closed state; permitted qualifiers: is:open, is:closed; examples: "roadmap is:open", "is:open feature planning". (string, optional)

- **update_project_item** - Update project item
- `item_id`: The unique identifier of the project item. This is not the issue or pull request ID. (number, required)
Expand Down
10 changes: 9 additions & 1 deletion pkg/github/__toolsnaps__/list_project_fields.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,14 @@
"description": "List Project fields for a user or org",
"inputSchema": {
"properties": {
"after": {
"description": "Forward pagination cursor from previous pageInfo.nextCursor.",
"type": "string"
},
"before": {
"description": "Backward pagination cursor from previous pageInfo.prevCursor (rare).",
"type": "string"
},
"owner": {
"description": "If owner_type == user it is the handle for the GitHub user account. If owner_type == org it is the name of the organization. The name is not case sensitive.",
"type": "string"
Expand All @@ -19,7 +27,7 @@
"type": "string"
},
"per_page": {
"description": "Number of results per page (max 100, default: 30)",
"description": "Results per page (max 50)",
"type": "number"
},
"project_number": {
Expand Down
16 changes: 12 additions & 4 deletions pkg/github/__toolsnaps__/list_project_items.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,19 @@
"title": "List project items",
"readOnlyHint": true
},
"description": "List Project items for a user or org",
"description": "Search project items with advanced filtering",
"inputSchema": {
"properties": {
"after": {
"description": "Forward pagination cursor from previous pageInfo.nextCursor.",
"type": "string"
},
"before": {
"description": "Backward pagination cursor from previous pageInfo.prevCursor (rare).",
"type": "string"
},
"fields": {
"description": "Specific list of field IDs to include in the response (e.g. [\"102589\", \"985201\", \"169875\"]). If not provided, only the title field is included.",
"description": "Field IDs to include (e.g. [\"102589\", \"985201\"]). CRITICAL: Always provide to get field values. Without this, only titles returned. Get IDs from list_project_fields first.",
"items": {
"type": "string"
},
Expand All @@ -26,15 +34,15 @@
"type": "string"
},
"per_page": {
"description": "Number of results per page (max 100, default: 30)",
"description": "Results per page (max 50)",
"type": "number"
},
"project_number": {
"description": "The project's number.",
"type": "number"
},
"query": {
"description": "Search query to filter items",
"description": "Query string for advanced filtering of project items using GitHub's project filtering syntax.",
"type": "string"
}
},
Expand Down
14 changes: 11 additions & 3 deletions pkg/github/__toolsnaps__/list_projects.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,17 @@
"title": "List projects",
"readOnlyHint": true
},
"description": "List Projects for a user or org",
"description": "List Projects for a user or organization",
"inputSchema": {
"properties": {
"after": {
"description": "Forward pagination cursor from previous pageInfo.nextCursor.",
"type": "string"
},
"before": {
"description": "Backward pagination cursor from previous pageInfo.prevCursor (rare).",
"type": "string"
},
"owner": {
"description": "If owner_type == user it is the handle for the GitHub user account. If owner_type == org it is the name of the organization. The name is not case sensitive.",
"type": "string"
Expand All @@ -19,11 +27,11 @@
"type": "string"
},
"per_page": {
"description": "Number of results per page (max 100, default: 30)",
"description": "Results per page (max 50)",
"type": "number"
},
"query": {
"description": "Filter projects by a search query (matches title and description)",
"description": "Filter projects by title text and open/closed state; permitted qualifiers: is:open, is:closed; examples: \"roadmap is:open\", \"is:open feature planning\".",
"type": "string"
}
},
Expand Down
73 changes: 73 additions & 0 deletions pkg/github/instructions.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,79 @@ Check 'list_issue_types' first for organizations to use proper issue types. Use
return `## Discussions

Use 'list_discussion_categories' to understand available categories before creating discussions. Filter by category for better organization.`
case "projects":
return `## Projects

Workflow: 1) list_project_fields (get field IDs), 2) list_project_items (with pagination), 3) optional updates.

Field usage:
- Call list_project_fields first to understand available fields and get IDs/types before filtering.
- Use EXACT returned field names (case-insensitive match). Don't invent names or IDs.
- Iteration synonyms (sprint/cycle) only if that field exists; map to the actual name (e.g. sprint:@current).
- Only include filters for fields that exist and are relevant.

Pagination (mandatory):
- Loop while pageInfo.hasNextPage=true using after=pageInfo.nextCursor.
- Keep query, fields, per_page IDENTICAL on every page.
- Use before=pageInfo.prevCursor only when explicitly navigating to a previous page.
- Do not analyze until ALL pages fetched.

Counting rules:
- Count items array length after full pagination.
- If multi-page: collect all pages, dedupe by item.id (fallback node_id) before totals.
- Never count field objects, content, or nested arrays as separate items.
- item.id = project item ID (for updates/deletes). item.content.id = underlying issue/PR ID.

Summary vs list:
- Summaries ONLY if user uses verbs: analyze | summarize | summary | report | overview | insights.
- Listing verbs (list/show/get/fetch/display/enumerate) → enumerate + total.

Self-check before returning:
- Paginated fully
- Dedupe by id/node_id
- Correct IDs used
- Field names valid
- Summary only if requested.

Return COMPLETE data or state what's missing (e.g. pages skipped).

list_project_items query rules:
Copy link
Contributor

@kerobbi kerobbi Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section (lines 123 - 156) on query construction is quite long. What I would suggest doing here is just keeping anything which is projects specific or that we think that the model does not know already, and removing the rest.

We do have a query parameter in other tools (e.g., search_pull_requests) that could be used as a reference in case you want to update the list_project_items query param. In general the model is quite good at constructing queries so it probably won't need this level of guidance.

If we were to consider adding guidance on search syntax in the future, that would apply across the entire server as a whole, not just projects tools, so it wouldn't live in these projects specific instructions.

Copy link
Contributor Author

@tmelliottjr tmelliottjr Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is worth validating again, but I did not have much success at all without introducing these query rules for projects. Projects search is also a combination of some issues search capabilities and project specific rules for interacting with items and their fields.

I also found, while working through enabling this for chat, that in its current form, chat failed spectacularly at using the tools at the right time and effectively didn't apply any filters.

Copy link
Contributor

@kerobbi kerobbi Nov 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chat failed spectacularly at using the tools at the right time

Having some server instructions to cover the ways the tools can/should be used together, i.e., workflows, should definitely help with this, although without running evals it's pretty much impossible to concretely determine if these changes would have any significant impact. We already have a couple of examples for other toolsets.

effectively didn't apply any filters

A bit hard to pinpoint which descriptions/instructions could be leading to this, as even a couple of words could make a huge difference, but just trying to take a step back here. Could the following base server instructions be a contributing factor? I'm wondering whether adding one or two sentences as part of the server instructions for projects, specific to the list_projects and list_projects_items, highlighting that we do not just want "basic filtering" can improve query building.

Another thought

If the somewhat right tool calls do happen, but the provided query filters are irrelevant or wrong, could it be that the model(s) is understanding or trying to do something entirely different? Is the difference between Projects (i.e., (the project management feature) and, for instance, repositories, clear enough to the models given the current descriptions and server instructions?

This is just me assuming that everything needed to query/filter projects is available as part of this documentation, and if that is the case, the model should already be aware of how to build queries.

Copy link
Contributor Author

@tmelliottjr tmelliottjr Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These instructions effectively only exist as part of the user/assistant flow which seems to work well enough for say, chat in VSCode. They do not appear to have much effect in dotcom chat.

I have tried running evals locally, but without success. I'll need to try again next week. I also tried to see if there were any ways to run evals with custom branches, but that didn't seem possible either.

In regards to the docs availability, how can I confirm that it is using those docs in context during any given tool call?

Query string - For advanced filtering of project items using GitHub's project filtering syntax:

MUST reflect user intent; strongly prefer explicit content type if narrowed:
- "open issues" → state:open is:issue
- "merged PRs" → state:merged is:pr
- "items updated this week" → updated:>@today-7d (omit type only if mixed desired)
- "list all P1 priority items" → priority:p1 (omit state if user wants all, omit type if user specifies "items")
- "list all open P2 issues" → is:issue state:open priority:p2 (include state if user wants open or closed, include type if user specifies "issues" or "PRs")
- "all open issues I'm working on" → is:issue state:open assignee:@me

Query Construction Heuristics:
a. Extract type nouns: issues → is:issue | PRs, Pulls, or Pull Requests → is:pr | tasks/tickets → is:issue (ask if ambiguity)
b. Map temporal phrases: "this week" → updated:>@today-7d
c. Map negations: "excluding wontfix" → -label:wontfix
d. Map priority adjectives: "high/sev1/p1" → priority:high OR priority:p1 (choose based on field presence)
e. When filtering by label, always use wildcard matching to account for cross-repository differences or emojis: (e.g. "bug 🐛" → label:*bug*)
f. When filtering by milestone, always use wildcard matching to account for cross-repository differences: (e.g. "v1.0" → milestone:*v1.0*)

Syntax Essentials (items):
AND: space-separated. (label:bug priority:high).
OR: comma inside one qualifier (label:bug,critical).
NOT: leading '-' (-label:wontfix).
Hyphenate multi-word field names. (team-name:"Backend Team", story-points:>5).
Quote multi-word values. (status:"In Review" team-name:"Backend Team").
Ranges: points:1..3, updated:<@today-30d.
Wildcards: title:*crash*, label:bug*.
Assigned to User: assignee:@me | assignee:username | no:assignee

Common Qualifier Glossary (items):
is:issue | is:pr | state:open|closed|merged | assignee:@me|username | label:NAME | status:VALUE |
priority:p1|high | sprint-name:@current | team-name:"Backend Team" | parent-issue:"org/repo#123" |
updated:>@today-7d | title:*text* | -label:wontfix | label:bug,critical | no:assignee | has:label

Never:
- Infer field IDs; fetch via list_project_fields.
- Drop 'fields' param on subsequent pages if field values are needed.`
default:
return ""
}
Expand Down
Loading
Loading