Rendered at 23:46:45 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
p1llus 8 hours ago [-]
One thing I found that I thought was a fun addition, is using eBPF made this even easier. No need to fight with anyone that is using certificate pinning, mTLS or anything else, you just get the raw plaintext data straight of the wire (right before encryption and right after decryption) and works nicely for most of the agents and IDE's.
That will in practice give you everything from telemetry to prompts, and its funny to see just how much some of them collect/run that is not at all related to your own ask..
A handy alternative when certain applications tend to make it harder to apply a MiTM proxy and you can dump it straight into your own scripts/programs to filter out and store it in whichever format you want for more analysis.
kro 7 hours ago [-]
Out of curiosity: How?
They don't offload TLS to the kernel, do they? Most apps do it in userspace linked against openssl afaik.
Do you patch that lib? If ebpf "just" operates at network/packet level, I don't see how it can do more than Mitmproxy in regard to avoid DH-PFS/Pinning
merb 6 hours ago [-]
It’s done via uprobes which allows ebpf to attach to users processes or libraries, basically like ld_preload but built into the kernel. First link that explains it https://blog.quarkslab.com/defeating-ebpf-uprobe-monitoring.... but there are many more links about it and it is a pretty useful tool for debugging in prod environments. (Also first link: https://blog.px.dev/ebpf-function-tracing/) this also the intended use case.
px.dev is btw a cncf project which also helps for ebpf debugging in k8s (comes from new relic)
p1llus 1 hours ago [-]
Exactly this :)
You can either attach the uprobes directly to the systems OpenSSL to capture anything using it (curl etc), though most of these LLM clients and similar uses their own static compiled BoringSSL, but its very straight forward to find, it does require some knowledge to determine the right place where you attach the uprobe, but after that it can easily be scripted to automatically find the right offsets (as these are usually stripped binaries)
orev 7 hours ago [-]
eBPF started as a network tool, but (according to the creators of it) targeting the network was just a ploy to get a foot in the door so they could start expanding it to other things. They didn’t think the idea would be accepted if they tried to do it all at once. So now it works on many parts of the system, not just the network.
somat 7 hours ago [-]
Surprised that works, I thought TLS was done entirely in process space.
I think I found it, My first thought was some sort of builtin ssl library backdoor, but it looks like you do some ld.preload shenanigans to inject a eBPF monitor. I am not sure exactly what the BPF brings to the table here. A convenient interface to intercept the accept() syscall?
p1llus 1 hours ago [-]
It is similar but no real shenanigans but rather a common functionality called uprobes. For most of these LLM clients which uses a static compiled OpenSSL or BoringSSL the functions you attach to is also the same overall, making it easily reusable when you know what to do. There has been plenty of srticles over the time on it, quite a fun thing to play around with. I made it work already with most of them for research purposes.
j0selit0 13 hours ago [-]
I was curious to understand how Copilot implements its harness, and also how I was exhausting my quota so quickly. End up going down a rabbit hole of intercepting its network traffic with mitmproxy.
A few interesting things I found along the way:
- watched model/capability discovery and routing happen in real time
- looked at what gets injected into context and sent with ghost completions
- found that recent edits can pull in context from files other than the one you're currently editing (including infamous .env)
- found the SQLite session store behind Chronicle, including previous prompts/responses
- watched the model query that history through tool calls
I then went through the VS Code source to reconcile some of what I was seeing on the wire with the actual implementation.
Overall some interesting lessons around how their harness is implemented.
jiehong 3 hours ago [-]
How do you actually cleanly solve that .env issue?
Anything cross platform and coding agent agnostic?
I suppose that .env file should be removed, but then things aren’t easy: no native multiplatform secret manager, or the std lib of the language doesn’t offer an API over the native secret store, etc.
Or a "secret injection proxy" for some cases could work I guess.
ElectricalUnion 1 hours ago [-]
Infisical, or Bitwarden Secret Manager? Those two look like perfectly reasonable if the llm is just careless (but still, nothing prevents the LLM from intentionally cat'ing /proc/self/environ or from running /usr/bin/env or set or similar)
theozero 59 minutes ago [-]
https://varlock.dev (free, open source) can pull secrets from many places, and has a credential broker (proxy) to inject placeholders, then replace with real secrets at the network boundary. There are a few other tools like this, but ours seems to be the most flexible so far.
Nice deep dive, I always wondered how copilot worked compared to similar tools. I'm shocked at the lack of of a rule for env files, I at least thought with a tool more integrated with github as a whole that would be a default but alas.
_davide_ 3 hours ago [-]
Disagree with the conclusion, even without carefully curated context every high end LLM perform just as well, maybe with an extra detour. In contrast if even one of the learnings is not up to date or doesn't apply to the current situation you find yourself with a long detour or even a failure.
Supermancho 7 hours ago [-]
I wish copilot was better at coding Java. It's like using ChatGpt 5.1, even with Fable 5 or Opus 5 as models.
The other issue with copilot is how episodic memory works. Copilot writes memories after a task is completed, which means a lot of context is lost from the intermediate exploration, success/failure steps (turns), for what? Codex's multithreaded model adds the turn outputs to episodic memory (both agents submit their episodic data to ... themselves for summary) which gives better insight when working on multi-step problems.
YawningAngel 2 minutes ago [-]
I have found beads works pretty well for this
bartek_gdn 10 hours ago [-]
Nice one! Really shows why we should run those in sandboxes without env access. I like the proxy swap approach
mathieu_aithos 8 hours ago [-]
Interesting to see how big companies make compromises with security for innovation and i feel that it's comprehensible and better that doing nothing.
But i guess it also show how we can see governance problems as real opportunity for involved peoples to build good systems with an agent native perspective.
personjerry 4 hours ago [-]
I believe this can be done without a MitM by using Wireshark or adjacent tool
jandrese 3 hours ago [-]
It's possible, but not easy. You need to export the internal state of your TLS stack to Wireshark in order for it to decrypt the traffic. This can be done via LD_PRELOAD type shenanigans but it's difficult and fragile.
MITM proxy is much easier to get working.
13 hours ago [-]
nottorp 7 hours ago [-]
"Apps users love, made with Electron". Seriously?
They use those apps either because of network effects or because there are few alternatives.
_davide_ 3 hours ago [-]
or completely deleting the app and use the browser version for desperation
That will in practice give you everything from telemetry to prompts, and its funny to see just how much some of them collect/run that is not at all related to your own ask..
A handy alternative when certain applications tend to make it harder to apply a MiTM proxy and you can dump it straight into your own scripts/programs to filter out and store it in whichever format you want for more analysis.
They don't offload TLS to the kernel, do they? Most apps do it in userspace linked against openssl afaik.
Do you patch that lib? If ebpf "just" operates at network/packet level, I don't see how it can do more than Mitmproxy in regard to avoid DH-PFS/Pinning
I think I found it, My first thought was some sort of builtin ssl library backdoor, but it looks like you do some ld.preload shenanigans to inject a eBPF monitor. I am not sure exactly what the BPF brings to the table here. A convenient interface to intercept the accept() syscall?
A few interesting things I found along the way:
- watched model/capability discovery and routing happen in real time - looked at what gets injected into context and sent with ghost completions - found that recent edits can pull in context from files other than the one you're currently editing (including infamous .env) - found the SQLite session store behind Chronicle, including previous prompts/responses - watched the model query that history through tool calls
I then went through the VS Code source to reconcile some of what I was seeing on the wire with the actual implementation.
Overall some interesting lessons around how their harness is implemented.
Anything cross platform and coding agent agnostic?
I suppose that .env file should be removed, but then things aren’t easy: no native multiplatform secret manager, or the std lib of the language doesn’t offer an API over the native secret store, etc.
Or a "secret injection proxy" for some cases could work I guess.
I dont see how you can ever really trust an LLM anyway to follow instructions.
The other issue with copilot is how episodic memory works. Copilot writes memories after a task is completed, which means a lot of context is lost from the intermediate exploration, success/failure steps (turns), for what? Codex's multithreaded model adds the turn outputs to episodic memory (both agents submit their episodic data to ... themselves for summary) which gives better insight when working on multi-step problems.
MITM proxy is much easier to get working.
They use those apps either because of network effects or because there are few alternatives.