Skip to content

Autoresearch: Execution engine optimization (large in-memory query) - #4348

Draft
dondonz wants to merge 3 commits into
masterfrom
claude/autoresearch-execution-large
Draft

Autoresearch: Execution engine optimization (large in-memory query)#4348
dondonz wants to merge 3 commits into
masterfrom
claude/autoresearch-execution-large

Conversation

@dondonz

@dondonz dondonz commented Mar 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

This is a bit of fun to see what Autoresearch can do

Summary

  • Autoresearch setup for optimizing LargeInMemoryQueryPerformance.benchMarkSimpleQueriesThroughput
  • Targets the core execution engine: ExecutionStrategy, ResultNodesInfo, field resolution, result assembly
  • 10M scalar field benchmark — cleanly isolates execution overhead
  • Designed for overnight autonomous optimization runs (200 iterations)

Approach

  • Profile-guided optimization using async-profiler
  • Targeted local test subset (graphql.execution.*) for fast iteration
  • Lessons from ENF autoresearch baked into program.md

Test plan

  • Run autoresearch loop locally
  • Verify improvements with full test suite on EC2
  • Review and cherry-pick winning optimizations

🤖 Generated with Claude Code

@dondonz @claude
Add autoresearch setup for execution engine (large in-memory query) o…
024c34c 
…ptimization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Mar 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Test Report

Test Results

Java Version Total Passed Failed Errors Skipped
Java 11 5927 (±0) 5871 (±0) 0 (±0) 0 (±0) 56 (±0)
Java 17 5927 (±0) 5870 (±0) 0 (±0) 0 (±0) 57 (±0)
Java 21 5927 (±0) 5870 (±0) 0 (±0) 0 (±0) 57 (±0)
Java 25 5927 (±0) 5870 (±0) 0 (±0) 0 (±0) 57 (±0)
jcstress 32 (±0) 32 (±0) 0 (±0) 0 (±0) 0 (±0)
Total 23740 (±0) 23513 (±0) 0 (±0) 0 (±0) 227 (±0)

Code Coverage (Java 25)

Metric Covered Missed Coverage vs Master
Lines 30047 3121 90.6% ±0.0%
Branches 8785 1519 85.3% ±0.0%
Methods 8026 1206 86.9% ±0.0%

No per-class coverage changes detected.

Full HTML report: build artifact jacoco-html-report

Updated: 2026-08-20 22:23:28 UTC

dondonz
dondonz commented Mar 22, 2026
View reviewed changes
Comment thread autoresearch-execution-large/autoresearch.sh Outdated
echo "--- Asking Claude to make an optimization ---"
CLAUDE_OUTPUT=$(claude \
--model sonnet \
--dangerously-skip-permissions \

@dondonz dondonz Mar 22, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yolo again!

I gave it a try with Opus and it was fine too

dondonz and others added 2 commits March 22, 2026 17:35
@dondonz @claude
Replace --dangerously-skip-permissions with scoped --allowedTools
5eca0de 
Use --permission-mode plan with explicit --allowedTools whitelist instead
of bypassing all permissions. The agent can read files, edit source code,
and run gradle (for profiling), but cannot run tests, git commit, or
perform other destructive operations. The outer harness handles those.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@dondonz
Allow git checkout in agent allowedTools
2db2c4c
@dondonz

dondonz commented Apr 3, 2026

Copy link
Copy Markdown
Member Author

Trying a similar concept with https://github.com/kunchenguid/gnhf (Autoresearch meets Ralph loop)

@dondonz

dondonz commented Apr 3, 2026

Copy link
Copy Markdown
Member Author

Results in master...gnhf/improve-performance-2fa5e3

@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Hello, this pull request has been inactive for 60 days, so we're marking it as stale. If you would like to continue working on this pull request, please make an update within the next 30 days, or we'll close the pull request.

@github-actions github-actions Bot added the Stale label Jun 3, 2026
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Hello, as this pull request has been inactive for 90 days, we're closing this pull request. We always welcome contributions, and if you would like to continue, please open a new pull request.

@github-actions github-actions Bot closed this Jul 4, 2026
@andimarek andimarek reopened this Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Stale

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dondonz @andimarek