Modified compute_init in nanochat/common.py to allow for CPU-only execution by checking the NANOCHAT_DEVICE environment variable. This enhances flexibility and hackability, aligning with the project's goals of accessibility.
The `KVCache.insert_kv` function was dynamically resizing the cache tensor whenever it ran out of space. This can be inefficient and lead to performance degradation, especially during long generation sequences.
This commit improves performance by pre-allocating a larger cache upfront, reducing the need for frequent resizing and minimizing memory reallocation overhead.
The previous implementation of `use_calculator` relied on a character whitelist to sanitize expressions before passing them to `eval`. This approach is brittle and can be bypassed.
This commit introduces two improvements:
1. The character whitelist check is now performed using a set for faster lookups.
2. A check has been added to explicitly disallow the use of double underscores (`__`) in expressions, which is a common vector for accessing sensitive attributes and methods in Python.
While this is not a complete solution for safely evaluating mathematical expressions, it significantly hardens the `use_calculator` function against common `eval` exploits.
