Version 0.10.0 (unreleased)¶

Important Changes¶

⚠️ The harp.typing package was renamed to harp.typedefs, so a first-party module no longer shadows the standard-library typing module. Update your imports.
⚠️ HTTP caching moved out of http_client into a dedicated http_cache application built on hishel 1.x. Configuration under http_client.cache.* must be migrated to http_cache.*, and several cache options no longer exist. See Changed below for the migration.
⚠️ harp-proxy config is now harp-proxy system config.
⚠️ The enable_ui setting was removed from the dashboard settings. Use enabled: false to disable the dashboard application entirely.
⚠️ The telemetry application was removed. HARP no longer sends anonymous usage statistics. A leftover telemetry: block in your configuration is now reported as configuration for an unloaded application, and is rejected under --strict.
⚠️ This release contains a database migration. blobs.id widens from 40 to 64 characters, because hishel 1.x keys cache entries with a sha256 hexdigest while message blobs use sha1, and both share that column. Existing rows are untouched and the migration is backward compatible.

HARP applies it for you. storage.migrate defaults to true, and on that default the proxy migrates to head while starting. Upgrading and starting 0.10 is enough, and running harp-proxy db:migrate up head beforehand is optional.

If you run storage.migrate: false, applying it is yours to do, and this one is not optional: until blobs.id is widened, any response the cache would store fails with a 500 instead of degrading to a cache miss. Traffic the cache does not store is unaffected. Run harp-proxy db:migrate up head before starting 0.10 on PostgreSQL or MySQL. The same applies if you want the schema changed before the process begins serving.

SQLite needs nothing, because SQLite does not enforce VARCHAR widths. A file-backed SQLite database is still migrated through alembic like any other backend.

Added¶

harp create project now accepts the project name as an argument and --no-app / --no-config flags to skip the application folder or the config file, running without interactive prompts when the name and git author (from git config) are available. New projects are created with an application folder by default.
harp create project now generates a config.yml documenting every available setting for each application (with default values and documentation links), fully commented out, instead of an empty file. The reference is introspected from the settings models, so it stays in sync with the code.
New make dev task in generated projects: starts the HARP proxy with auto-reload on file changes (watches the application package and/or config.yml), powered by watchfiles.
New harp-proxy system command group with config and services subcommands for system inspection:
- harp-proxy system config displays the compiled configuration (replaces harp-proxy config)
- harp-proxy system services shows all configured services with their dependencies, aliases and configuration
Application dependency resolution with topological sorting, so applications are initialized in the correct order.
ApplicationSettingsMixin for standardized enable/disable functionality in application settings.
autoload_dependencies parameter for ApplicationsRegistry.add() to automatically load declared dependencies.
--strict CLI flag for enforcing strict configuration validation.
Warning system for misconfigured applications (configuration exists for an application that is not loaded).
Two-pass configuration parsing to filter out applications with enabled: false.
Cache debugging headers on HTTP responses:
- X-Cache: MISS on non-cached responses
- X-Cache: HIT on cached responses
- Age, showing cache age in seconds for cached responses
Normalized cache key generation, so a resource reached through different backend servers shares one cache entry:
- Cache keys exclude protocol, hostname and port from URLs
- Implemented by a custom AsyncCacheTransport that normalizes URLs
harp.DEBUG flag for enabling debug information output, set via the HARP_DEBUG or DEBUG environment variables. When enabled, error responses include full exception tracebacks. Do not enable it in production, as it may expose sensitive information.
Integration test suites for the http_cache application (harp_apps/http_client/tests/with_http_cache/ and harp_apps/proxy/tests/with_http_cache/), covering cache behaviour, backward compatibility and isolation between endpoints.

Changed¶

HTTP client caching migrated from hishel 0.1.x to hishel 1.x, with a new architecture:
- New ``http_cache`` application: cache functionality moved to a dedicated http_cache application that depends on http_client, for a better separation of concerns.
- Package restructuring: cache code moved from harp_apps.http_client.contrib.hishel.* to harp_apps.http_cache.* (flattened structure).
- Configuration: cache configuration moved from http_client.cache.* to http_cache.*.
- Cache policy now uses hishel.SpecificationPolicy with CacheOptions instead of hishel.Controller, and the documentation examples use policy instead of controller.
- The following cache configuration parameters are no longer available, and this release provides no way to set them from configuration:
  - cacheable_status_codes, now determined by RFC 9111 compliance
  - cacheable_methods, whose behaviour is now CacheOptions.supported_methods
  - allow_stale, whose behaviour is now CacheOptions.allow_stale
  The CacheOptions instance is built by an internal service that no setting points at, so there is no supported replacement for these three in 0.10. If you relied on any of them, see #933, which tracks making them configurable again.
- allow_heuristics moved from the cache policy to the cache storage, since hishel 1.x has no equivalent option on the policy. Set http_cache.storage.arguments.allow_heuristics. It stays false, as it was before, and see Security below for why.
- A response whose only freshness information is a validator (ETag, Last-Modified) is no longer kept for conditional revalidation, so it is refetched in full rather than revalidated with a 304.
- Cache entries written by hishel 0.1.x are not reused. hishel 1.x derives a different cache key, so existing entries are never matched and the first request for each resource after the upgrade goes to the origin. Old rows are left in place and are not served; nothing is lost beyond a cold cache.
- If you use a custom cache configuration, update it:
  
  Old (hishel 0.1.x, under http_client.cache):
```
http_client:
  cache:
    enabled: true
    controller:
      type: hishel.Controller
      arguments:
        allow_stale: false
        cacheable_methods: [GET, HEAD]
```
  New (hishel 1.x, http_cache application):
```
http_cache:
  enabled: true
  policy:
    type: hishel.SpecificationPolicy
```
  The policy’s CacheOptions are fixed in this release: shared cache, GET and HEAD, and allow_stale: false. Any of the three removed options you were setting has no equivalent here, see #933.
- Note: http_cache uses a temporary workaround in its on_bind event to configure http_client’s transport. This will be replaced with proper cross-app service overrides once #806 is implemented.
The enable_ui setting was removed from DashboardSettings. The enabled field from ApplicationSettingsMixin now controls whether the dashboard application is loaded; use enabled: false to disable it.
harp-proxy config was renamed to harp-proxy system config. All existing options (--raw, --json, --unsecure) continue to work.
The harp.typing package was renamed to harp.typedefs. It shadowed the standard-library typing module and crashed the CLI whenever the harp/ directory ended up on sys.path.
The cookiecutter template migrated from Poetry to UV (PEP 621, Python 3.13, hatchling), with an enhanced Makefile, improved prompts and automatic git initialization.
harp create now runs cookiecutter as a subprocess, falling back to uv tool run cookiecutter when it is not installed as a package, and its guidance no longer mentions Poetry. This completes the UV migration for the project bootstrap tooling.
The published wheel no longer ships test code, snapshots or dev-only testing utilities, reducing the distribution size. The harp create project scaffolding, which legitimately contains tests, is preserved.
Controller signature resolution in the ASGI kernel is memoized instead of being recomputed on every request.
The SQL blob storage seen-cache is populated on write, removing a redundant existence query when the same blob is stored again.
get_transaction_list reads its total count and its page in a single session, so both share a consistent snapshot.
The Docker latest tag now follows every mainline stable release. It was pinned to 0.9.x, so cutting 0.10 would have left it on 0.9.
http-router is constrained to >=5.0,<6.0, so installing HARP resolves the major version the test suite runs against. The bound briefly allowed 6.0, which no HARP release has been tested with.
Dashboard form controls change colour. A selected checkbox or radio is now indigo rather than blue, and the borders of unselected controls are lighter. Most visible on the transaction list filters. This comes from @tailwindcss/forms 0.5.11, which stopped overriding colours the dashboard’s own components had always specified; nothing in the dashboard was restyled.

Fixed¶

http_cache: a stale cache entry is revalidated properly instead of being downloaded again in full. Two defects, fixed together because either alone leaves the cache worse than it is now:
- The conditional headers hishel adds for revalidation (If-None-Match, If-Modified-Since) never reached the origin, because the request actually sent was the one captured before they were added. The origin had no validator to compare against, so it answered 200 with the whole body at every staleness boundary rather than a small 304.
- A 304 freshened nothing, because AsyncStorage.update_entry did not resolve the entry hishel asked it to update. The entry stayed exactly as stale as it was and was revalidated on every subsequent request, without end.
Measured through a running proxy on PostgreSQL, three client requests across one staleness boundary: the origin is now asked twice and transfers the body once, where it previously transferred the body twice. Resources with no validator are unaffected.

This concerns responses the cache does retain, which means those the origin declared fresh. A response whose only freshness information is a validator is still not retained at all, for the reason given under Changed above, so there is nothing to revalidate for it.
The HTTP cache can store entries on PostgreSQL and MySQL again. blobs.id was declared 40 characters wide, the length of the sha1 hexdigest that message blobs use, while hishel 1.x keys cache entries with a sha256 hexdigest of 64. Both share the column, so every cache write was rejected by the database and the proxy returned a 500 instead of degrading to a cache miss. SQLite did not show it, because it does not enforce VARCHAR widths.
The published wheel now lists packaging in its Requires-Dist metadata. HARP imports it in harp/__init__.py, so any import harp needs it, but it was only ever installed as a transitive dependency of dataclasses-json. Nothing was broken and no version changed; what changed is that the requirement is now declared rather than inherited. Packagers and offline mirrors resolving from metadata should note the added entry. (This is a metadata correction, not a dependency bump, which is why it is recorded here.)
proxy: a cache hit recorded against a transaction now comes from the cache itself rather than from the response’s X-Cache header. The header is a public one that upstreams set for their own caches, so an origin behind a CDN (CloudFront, Fastly and Varnish all emit X-Cache) was recorded as a HARP cache hit whenever the http_cache application was not loaded, and the dashboard reported hits for a cache that was not running. Age is likewise measured by HARP’s own clock instead of being read back from the response. The X-Cache and Age response headers themselves are unchanged.
Empty or comments-only configuration files no longer crash configuration loading; they now load as an empty configuration.
Applications configured with enabled: false are now filtered out when the configuration is provided as Pydantic model instances, not only as dicts. They were previously still loaded, and their services still registered.
Restored the harp command as an alias of harp-proxy, so harp ... and uv run harp ... run the CLI instead of executing the package directory. The documented harp create project flow works again.
Committed a .gitkeep in harp_apps/dashboard/web/ so a fresh clone can uv sync and build the wheel. The directory is a hatchling force-include target and previously vanished on checkout, making the build fail with FileNotFoundError.
make wheel completes on machines that ship python3 but no python, which is most Linux distributions. It built the wheel and then exited 127 counting the bundled dashboard assets, so neither that check nor the twine check below it ever ran and the wheel was never validated. The interpreter is now taken from whichever branch built the wheel, since uv builds with its own managed interpreter and puts no python on PATH. CI did not catch this because actions/setup-python provides the python the developer machine lacks.
Requesting a transaction that does not exist now returns a 404 from the dashboard instead of an unhandled error. The storage layer raised NoResultFound, so the controller’s 404 handling was never reached.
The dashboard “Help” link opens the documentation for the version you are running. It was pinned to the 0.9 documentation, so every 0.10 user landed on the previous release’s docs. The URL is now derived from the reported version, so it follows future releases on its own.
The dashboard is navigable on a phone. Below the 640px breakpoint the menu button had no menu behind it, so Overview, Transactions and System could not be reached at all and the dashboard was a single dead-end page. The button also reported itself as expanded to assistive technology while controlling nothing. The header no longer draws the logo on top of the user and version text at those widths, and the active item is now marked with aria-current rather than by colour alone.
janitor: a sweep no longer leaves the SQL blob storage believing the blobs it just deleted still exist. Populating the seen-cache on write, new in this release, made that belief outlive the row, so every later write of the same content became a silent no-op, the HTTP cache could never repopulate after the first sweep, and Cache entry incomplete appeared in the logs. A cacheable response now costs one origin fetch per janitor sweep, as it did in 0.9.1, instead of one per request. The sweep still discards live cache entries every 600 seconds, which is inherited behaviour tracked in #923 and not something this release changes.
The HTTP cache created_at field is written and read as UTC. On a non-UTC host, the stored value was a local wall-clock time labelled GMT.
Proxy health-check tasks are now cancelled on shutdown. The on_shutdown handler was never registered, so the task group leaked on every teardown.
MySQL full-text search binds the search term as a value instead of a column expression, which broke the MATCH ... AGAINST path at runtime.
The dashboard keeps a reference to its initial user-creation task, which could otherwise be garbage collected before completion.
Replaced the dead GitLab CI/CD pipeline badge and stale “CI/CD” link in the README with the GitHub Actions workflow, aligning the public docs with where CI actually runs since 0.9.0.
Generated projects now properly isolate pytest tests and include correct startup instructions.
Fixed test discovery incorrectly including misc/ directory worktree applications in test_all_applications_settings.py.
uv.lock is now consistent with pyproject.toml: the redis requirement recorded in the lockfile had drifted from the manifest, so every uv sync on a clean checkout silently rewrote uv.lock and left developers with a dirty working tree they did not create.
CI now runs uv lock --check before any job that runs uv sync, so a pull request that updates pyproject.toml without relocking fails the gate instead of being reconciled in place and reported green.

Security¶

proxy: the upstream URL is now built from the path component of the incoming request only, and the request is refused with 400 Bad Request if it would resolve to another origin. A request path carrying a scheme (GET /http://example.org/) could previously make the proxy forward to an arbitrary host (CWE-918, server-side request forgery / open proxy).
proxy: an endpoint whose url carries a path exposes one subtree of an upstream. Dot segments in the request path could walk above that subtree, so GET /../../admin against an endpoint on http://upstream/api/v1/ reached http://upstream/admin. The origin was already checked and stayed correct, which is why nothing else in the chain noticed. Such a request is now refused with 400 Bad Request.
proxy: HARP forwarded a malformed request that a proxy must not forward. When a client sent both Content-Length and Transfer-Encoding, HARP passed the client’s Content-Length to the upstream while sending a body of a different length, so the upstream was told to read bytes that were never sent and a pooled connection could desynchronise. HARP now states the framing of the body it actually sends.
proxy: connection-specific headers (Connection and the fields it names, Keep-Alive, TE, Trailer, Upgrade, Proxy-Authenticate, Proxy-Authorization) were forwarded instead of being dropped at the hop, as RFC 9110 §7.6.1 requires of an intermediary. This applies in both directions: to the request sent upstream, and to the response returned to the client, including a response replayed from the cache. On the response side the fields named by Connection are dropped by http_cache at the point the upstream answer arrives, which is the last moment Connection is still readable, so they are never stored either (RFC 9111 §3.1).
http_cache: a shared cache could serve one caller’s response to another. Heuristic freshness, which lets a cache invent an expiry from Last-Modified for a response the origin never declared cacheable, was disabled in HARP up to 0.9.1 and became active when the HTTP client migrated to hishel 1.x, which removed the option. Since the cache key is derived from the request URL alone, an upstream that authenticates callers by anything other than the Authorization header could have responses reused across them. It is off again by default, and opt-in via http_cache.storage.arguments.allow_heuristics.
http_cache: a shared cache no longer retains responses it must not share. Up to 0.9.1 a response marked private was stored in the shared cache, and a response to a request carrying Authorization was served to the next caller regardless of their credentials. Measured against 0.9.1, one caller’s authenticated response body was returned verbatim to a second caller presenting a different token. Both are now declined, and the behaviour is covered by tests.
config: credential masking did not work in the mode most callers use. asdict(secure=True) recognised a password by the type of the value holding it, and pydantic renders a URL to a plain string before that check is reached, so the masking silently did nothing. Passwords are now masked wherever they appear, including in settings that are plain strings rather than typed URLs. This affected harp-proxy system config, which printed database and Redis passwords in full under --raw and --json, and never masked anything without a flag; and the dashboard’s GET /api/system/proxy, which exposed credentials embedded in an upstream endpoint url on an API that is unauthenticated unless dashboard.auth is configured.
http_cache: a client’s Cache-Control: no-store no longer prevented the response being cached. RFC 9111 §5.2.1.5 makes this a MUST: a cache must store no part of such a request or of any response to it. hishel 0.1.x checked the request directive, hishel 1.x decides cacheability from the response alone, and the check was lost when HARP migrated, so a caller asking not to be retained was retained and the next caller was served that entry. Measured on PostgreSQL, two identical no-store requests reached the origin once instead of twice. Such a request now bypasses the cache entirely: nothing is stored, and an existing entry is not used to answer it. The response directive (§5.2.2.5) was already honoured and is unchanged. Request no-cache keeps forcing revalidation.

This binds HARP’s cache only. The transaction record is a separate store and the directive does not reach it, so a request carrying no-store is recorded like any other, as far as recording gets under the load you are running. See What HARP retains, and what a caller can do about it, and Known Limitations below.
config: !cfg comparison operands are parsed as Python literals instead of being evaluated as code. Configuration is author-time and trusted, so this is defense in depth.

Removed¶

Removed the telemetry application, which was sending anonymous usage statistics every 24 hours. This reduces network overhead and eliminates potential privacy concerns for users in regulated environments.

Known Limitations¶

Not changes in this release. Carried into it, and worth knowing before you deploy.

A resource that varies gets little or no cache reuse. Vary is honoured before an entry is reused, so no caller is ever served a variant that does not match their request. But the storage holds one entry per cache key, so two variants of the same resource overwrite each other, and a caller alternating between them gets no cache hits at all. This is not new in 0.10: the behaviour is the same on 0.9.1. Tracked in #910.
Transaction recording is best-effort, and under load it records a fraction of your traffic. When the storage worker falls behind it sheds message detail first and then whole transactions, rather than applying backpressure and slowing traffic down. Nothing warns you that it happened: no counter, no log line, and a shed payload looks in the dashboard exactly like a request that carried none. In a measured sweep on PostgreSQL at concurrencies from 5 to 60, roughly 80% of transactions were never stored and two thirds of the rows that were stored held no messages at all. Proxying stayed healthy throughout, so the traffic was served correctly and only the record of it was lost. Do not treat the transaction record as a complete account of what passed through the proxy. This is not new in 0.10, and the measurements and their exact conditions are in #945.