mirror of
https://github.com/wshobson/agents.git
synced 2026-03-18 17:47:16 +00:00
fda45604b7
* ⚡ Bolt: Reuse ThreadPoolExecutor in PromptOptimizer 💡 What: Initialized `ThreadPoolExecutor` in `PromptOptimizer.__init__` and reused it in `evaluate_prompt`. 🎯 Why: The previous implementation created a new `ThreadPoolExecutor` for every call to `evaluate_prompt`. Since `evaluate_prompt` is called repeatedly inside the `optimize` loop (and for every variation), this caused significant overhead from repeatedly creating and destroying thread pools. 📊 Impact: Benchmark showed a reduction in execution time from ~5.36s to ~3.76s (~30% improvement) for 500 iterations with a mocked LLM. 🔬 Measurement: Ran a benchmark script executing `evaluate_prompt` 500 times. Before: 5.36s After: 3.76s * ⚡ Bolt: Reuse ThreadPoolExecutor in PromptOptimizer 💡 What: Initialized `ThreadPoolExecutor` in `PromptOptimizer.__init__` and reused it in `evaluate_prompt`. Added a `shutdown` method for proper cleanup. 🎯 Why: The previous implementation created a new `ThreadPoolExecutor` for every call to `evaluate_prompt`. Since `evaluate_prompt` is called repeatedly inside the `optimize` loop (and for every variation), this caused significant overhead from repeatedly creating and destroying thread pools. 📊 Impact: Benchmark showed a reduction in execution time from ~5.36s to ~3.76s (~30% improvement) for 500 iterations with a mocked LLM. 🔬 Measurement: Ran a benchmark script executing `evaluate_prompt` 500 times. Before: 5.36s After: 3.76s * ⚡ Bolt: Reuse ThreadPoolExecutor in PromptOptimizer 💡 What: Initialized `ThreadPoolExecutor` in `PromptOptimizer.__init__` and reused it in `evaluate_prompt`. Added a `shutdown` method and wrapped execution in `try...finally` for proper resource management. 🎯 Why: The previous implementation created a new `ThreadPoolExecutor` for every call to `evaluate_prompt`. Since `evaluate_prompt` is called repeatedly inside the `optimize` loop (and for every variation), this caused significant overhead from repeatedly creating and destroying thread pools. 📊 Impact: Benchmark showed a reduction in execution time from ~5.36s to ~3.76s (~30% improvement) for 500 iterations with a mocked LLM. 🔬 Measurement: Ran a benchmark script executing `evaluate_prompt` 500 times. Before: 5.36s After: 3.76s --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
4 lines
383 B
Markdown
4 lines
383 B
Markdown
## 2024-05-23 - Thread Pool Overhead in Iterative Tasks
|
|
**Learning:** Recreating `ThreadPoolExecutor` inside a frequently called loop (like an optimization loop) introduces significant overhead, especially when the individual tasks are short-lived.
|
|
**Action:** Initialize `ThreadPoolExecutor` once in the class `__init__` and reuse it across method calls to amortize the setup cost.
|