Mum, I'm auditing a math heavy codebase, what do I do ?

I spent 3 days including saturday and sunday deriving formulas from the Gyro ECLP whitepaper. Line by line. Variable by variable. By Sunday night, I understood the math beautifully, and had found exactly zero bugs.
That weekend taught me something important: understanding math-heavy code and auditing it efficiently are two completely different skills. You can be a brilliant mathematician and a terrible math auditor. The reverse is also true.
This guide is everything I wish I knew before that weekend. It's the methodology that emerged from auditing Balancer, Gyro, Lido BLS, and various fixed-point libraries, including what worked, what didn't, and where I wasted time so you don't have to.

Triage — Is This Actually Math-Heavy?
Before you burn 40 hours trying to understand a Newton-Raphson approximation that's been audited at least six times, you need to know what you're actually looking at. Not all "math code" is created equal, and the first 30 minutes of any audit should answer one question: how deep does this go?
The "Can I Understand This In 30 Minutes?" Test
Open the core math file. Set a timer. Try to trace a single calculation end-to-end, do it really, do not ask an AI to do it for you.
If after 30 minutes you can explain what the function computes in plain English, identify the precision/decimal scheme being used or spot where rounding happens you're dealing with standard fixed-point arithmetic. These audits follow predictable patterns: check rounding directions, verify overflow bounds, test edge cases, you can move to systematic review.
If after 30 minutes you still don't know what
dSq3means, see precision constants like1e38alongside the usual1e18or even find yourself googling "ECLP invariant" or "Bhaskara's formula", you're in deep math territory. Stop. This is where the what to do and what to not do begins.
The honest version of this test: Can you explain to another auditor why this formula is correct? . If you can't, you don't understand it yet and you need a methodology to understand it first and to audit it in second.
Deep Math Ahead , am I lost ?
// === XOF-style hash chaining ===
let j := b // Pointer to next position in output chain
for { let i := 2 } 1 { } {
// XOR `b0` with previous output and hash it
mstore(s, xor(b0, mload(j)))
j := add(j, 0x20)
mstore(j, sha2(s, sub(dstPrime(add(0x20, s), i), s))) // SHA2 with DST index `i`
i := add(i, 1)
if eq(i, 9) { break } // Loop from i = 2 to i = 8 (7 iterations)
}
First, don’t panic, remember everyone is feeling approximately the same against this kind of code, if you don’t understand anything in the first 2 hours, it's OK, this code wasn't written in 2 hours either. Take your time, use AI to help, if you spend quality time in this piece of code, you will get it.
The bugs here aren't just "forgot to round up." They're "the formula is subtly wrong" or "error propagation wasn't bounded correctly".
I usually try to classify in 3 categories the type of math I’m facing: AMM/Invariant Math first , cryptographic math second and financial Math third. (Obviously if you’re facing one category, it doesn’t mean another one is not hiding) because you have to know and I’m sure you already know that different math types have different bug patterns. Let’s give some examples:
AMM / Invariant-Based Math ⇒ Examples: Curve StableSwap, Balancer Weighted/Stable pools, Gyro ECLP where you have to expect conservation invariants, price calculations derived from invariant equations or even heavy use of directional rounding (
mulUp/mulDown,divUp/divDown,powUp,powDown).Cryptographic Math ⇒ Examples: BLS signatures, Merkle proofs, elliptic curve operations where you have to expect precompile calls (addresses like
0x05or0x09for EIP-2537 BLS), constant-time requirements (or lack thereof) or even spec compliance requirements (EIPs, RFC documents).Financial Math ⇒ Examples: Interest rate calculations, yield accumulators, fee computations where you have to expect time-weighted calculations (often using accumulators), compounding, exponential operations or even precision loss over many operations.
The answer to "What kind of math is this?" determines your audit approach. AMM math needs rounding analysis. Crypto math needs spec diffing and financial math needs long-horizon and boundaries testing. Though all of them should almost always have boundaries (0,max value) and extreme number guardrails.
It is in fact Math Heavy, what to do now ?
The instinct is to dive straight into the code. Fight it. For math-heavy contracts, the research phase isn't optional prep work, it's where you find half your bugs. Moreover if you go straight on the code you will convince yourself that the code is good because you do not understand what’s the real purpose. A 2-year old vulnerability in PRBMath, one of the most widely-used fixed-point libraries in Solidity, went undetected despite comprehensive unit testing and multiple eyes on the code. The researchers who found it didn't even need to understand the bit-hacking implementation. They compared behavior against a specification.
That's the power of proper research: you can find bugs in code you don't fully understand.
When to Research vs. When to Dive In
Research first when there are resources available, here GPT and OPUS 4.5 deep research are your friends, in the contrary dive in first when it’s a standard algorithm and self contained code.
A practical heuristic (at least for me): if the NatSpec comments reference a mathematical paper or specification, read that document or at least skim it before you read another line of code. The StableMath.sol file in Balancer literally links to Curve's StableSwap whitepaper in the comments. That whitepaper tells you how the invariant should behave , and what it means when it doesn't.
Never forget to “solodit” it , usually GPT finds the vulnerabilities related to what you’re auditing but for me “soloditing” a topic should be as known as “googling” something, should be your first reflex.
\*Another tip*: When the code you're auditing is derived from a reference implementation, your first analysis should be a diff: core math functions , rounding directions choices, precision constants, error handling, this is usually where you start questionning your own understanding of the lines you’re reading because you’re seeing different explanations of it.
The Research Mindset
Research isn't about becoming an expert in elliptic curves or Newton-Raphson before you start. It's about building a reference frame for evaluating the code.
When you've read the Curve whitepaper, you know that the StableSwap invariant should be symmetric in the token balances. When the code isn't symmetric, that's suspicious.
When you've seen the PRBMath disclosure, you know that signed arithmetic rounding is tricky. When you see signed fixed-point math, you check the rounding direction.
When you've searched Solodit and found that every TWAMM audit mentions overflow in long-term order accumulators, you know exactly where to start your analysis.
One of the good ways for me to really understand something is to see it in different format. For ZK for example, I watched the ZKP MOOC (I did understand around 5% of the infos in it), then I did the ZK bootcamp from Rareskills (reaching 50/60%) , and finally I bought David Wong book RealWorld cryptography where everything was explained in a different manner than before but everything started to make sense. So my own personal advice is try to find different ways to understand your difficult topic. The best way will always be to teach/explain the thing you’re trying to understand of course, this is where you see where you’re lacking knowledge.
The research phase turns "I need to understand everything" into "I need to check these specific things." That's how you audit efficiently that’s the state of mind you should aim for before going on a phase when you’re reaching the “attacker mindset” part of your audit.
The Understanding Phase

These are good tips, but how TF do I reach the “I understand the code” phase ? …”
You can't audit math you don't understand. But "understanding" doesn't mean becoming a mathematician, it means being able to predict what the code should do before you read what it does do, it means being able to say “this line is important,what if something is going wrong before, what could go wrong if this goes wrong ?”
Why Visual Notes Beat Mental Models
Complex math has too many moving parts to hold in your head. When Gyro's ECLP calculates calcAChiAChiInXp, you're tracking: two precision scales (1e18 and 1e38), rotation parameters (c, s, lambda) , derived parameters (u, v, w, z, dSq) ,error bounds propagating through each step.
Please write it down. Draw the data flow. Sketch the geometric intuition if there is one, write what are the variables useful to understand, what are the variables we don’t really care about, try to always understand the purpose of a specific calculations, having in mind the why of something will always make easier understanding the how.

So…..what works ? for me my remarkable is the best but a pen and a paper is also very good for quick diagrams, re-deriving formulas. Desmos/Geogebra of course for visualization and wolfram alpha for checking algebraic manipulations. If you reach a point where you visualize the code you’re seeing you’ve made a major step toward auditing it.
The physical act of writing/drawing forces you to be explicit about each step. That's where you catch the gaps in your understanding.
Try to always have a “Symbolic Execution” moment in your understand/audit phase, this is where you simplify everything, you only keep the decimals, the directions your computation should get, you remove all the difficulty and just verify a simple (but really important) things, for example:

What "Deep Understanding" Actually Means
You understand the code when you can answer:
What invariant does this preserve? (For AMMs: what stays constant across swaps?)
What should happen at the boundaries? (Zero liquidity, max amounts, extreme prices)
Who benefits from rounding in each direction? (Protocol vs. user, LP/Liquidator vs. trader)
What would break this? (If you can't think of attack vectors, you don't understand it yet) , remind the famous “question it until you break it “
The test: If you can answer all four, you're ready to break it.
Systematic Vulnerability Hunting by Math Type

Now you understand the math. Time to break it.
AMM / Invariant-Based Math: "Every Incorrect Rounding Is a Bug" (please read this wonderful article from Josselyn Feist). For each math operation, ask: "If this rounds 10,000 times, who profits?" remember that round in favor of the protocol is no longer sufficient, make a rounding table (see appendix) detailing each step of the formulas you’re reviewing to see if the rounding directions are always the good choices and remember 2 important things that can happen here: trapping problem (rounding in favor of the protocol can also create problems for the protocol, overflow for example) and intermediate value dilemma (for complex formula we need to be sure of each rounding direction taken during the computation).
Key question: "How should this invariant behave at the extremes, and what rounding direction preserves that behavior during the computation ?"
Cryptographic Math: Build a spec compliance checklist (encoding, domain separation, edge cases), find the deviations between the precompile calls against the EIP, understand and know the attack patterns (subgroup attacks, malleability, timing leaking secrets).
Key question: "Does this exactly match the spec, byte-for-byte? How much of the trust is forwarded to the precompiles ?"
Financial Math: This is where your imagination has all its place, test extreme values: 0, 1 wei, max
uint256, 10e-18, 10e18, MIN,MAX , 0 block, 1 block, 5 years, decimal mismatch, epoch boundaries. You also have behavior to think about, like pools with very low liquidity or extreme imbalance enable rounding to zero (drain attacks), price manipulation with minimal capital,overflow in accumulator-style calculations, flash loan, fee computation, for a pool always test with 1 wei of token0 and maximum of token1.Key advice: basically, fuzz it with your head or with a tool.
Helping tools & workflows
Visualization Tools: Desmos/Geogebra, Wolfram Alpha, Python is also your best friend with matplotlib
Fuzzing: focus on round trip, monotonicity, boundary behaviors, this is where Medusa, Echidna and Alex The Entreprenerd are your best friends.
SMT Solvers: Certora Prover is the best but you can still use your own Z3 via python API, Halmos, K Framework
Pen and Paper: I personnaly love my Remarkable tablet but a pen and paper are always your first best friends
What NOT to Focus On (Time-Wasters)
Audit time is finite. Knowing what to skip is as important as knowing what to check.
Don’t get trapped into a formula derivation for too long, always try to estimate the value of the time you’re spending, from my experience reviewing ECLP pools, i spent an entire week end on the ECLP document deriving each formulas, i wasn’t even looking at the code and to be honest when i read the code after it made a lot more sense but sometimes the specs are really different than the real implementation due to the nature of the language (like solidity limitations) which doesn’t allow what the specifications would perfectly want, for this type of case, deriving the formulas is not that useful, this is something really personal because for someone it will be valuable and for someone else only reading the code is enough so it’s a hard question, you always need to estimate the value of the time, a good indicator is to feedback yourself every 2 hours: “were the last two hours valuable / will they be valuable ?”.
When to Stop Understanding and Start Trusting
You don't need to understand every line of LogExpMath.sol. Those 300 lines of precomputed constants and bit manipulation implement exp() and ln() functions with decades of mathematical foundation.
Trust (but verify): well-audited libraries (Balancer's FixedPoint, OpenZeppelin's Math), standard algorithms with extensive test coverage.
and Don't trust: Forks with modifications (diff them), "optimized" versions that changed the math, anything the current team wrote themselves. Remember to try Formal Verification.
Common Time Sinks
Over-analyzing converged code. Newton-Raphson implementations that have run millions of iterations in production probably converge correctly. Check the bounds and tolerance, then move on.
Chasing dust. Precision loss of 1 wei per transaction is often acceptable. Calculate the cumulative impact: 1 wei × 1 million transactions = 10¹² wei = 0.000001 ETH. Remember that when doing tests, fuzzing, manual computation using Python it is always good to start with a fresh state of mind and remove boundaries, fees and guardrails on the code, it should almost always lead to medium/low findings but once you find them and you evaluated the severity in the worst case scenario, remember there is gas, there are fees, there are boundaries the maths can’t go.
Auditing the audited. If Curve's StableSwap math has been reviewed 50 times and the code is unchanged, your time is better spent on the integration than re-auditing the invariant calculation.
Proving negatives. Sometimes you suspect a bug but can't find it. Set a time limit. If you've spent 2 days and can't construct an attack, document your analysis and move on. You can return later with fresh eyes but always share it with the protocol, serious protocol will value it.
Closing Thoughts
Math-heavy audits are intimidating because the bugs hide in places most auditors don't look: off-by-one errors in rounding direction, overflow conditions that only trigger after years of operation, edge cases at extreme but realistic parameters.
The auditors who find these bugs share a few traits: they do the research before touching code, they work through the math by hand, they know what to check for each math type, and they know when to stop chasing rabbit holes.
This isn't about being a mathematician. It's about having a systematic approach and the discipline to follow it. Regarding my ECLP week end spent, I should have skimmed the doc first , understand what the invariant should preserve (ellipse geometry and price bounds), not how it’s computed and derived because then the solidity code was a lot different (err boundaries, not real calculations due to solidity limitations), identify trust boundaries (LogExpMath.sol battle tested for example). I should have started writing first something like a rounding table because it would have made my understanding faster, a “which variable is used for” directly because this is (for me) the moment i’m starting to really learn.
Remember every incorrect rounding is a bug. Check the worst case first, always trace rounding bias from output back to each intermediate step. And when the numbers get too big to hold in your head, write them down.
Thank you for reading!
Appendix
Rounding Table for Gyro ECLP function calculateInvariantWithError:
| Line | Operation | Direction | Purpose |
| 340 | calcAtAChi(...) | — | Compute At·Aχ (error compensated) |
| 341 | calcInvariantSqrt(...) | sqrt: ↓, err: ↑ | Discriminant sqrt + error bound |
| 346 | (err + 1).divUpMagU(2 * sqrt) | ↑mag | Error propagation through sqrt |
| 350 | GyroPoolMath.sqrt(err, 5) | ↓ | Fallback error (compensated by scaling) |
| 353 | ((λ*(x+y)/_ONE_XP) + err + 1) * 20 | ↑ net | Scale numerator error conservatively |
| 357 | _ONE_XP.divXpU(AχAχ - 1) | ↓ | 1/denominator (underestimate) |
| 361 | (numerator).mulDownXpToNpU(mulDenom) | ↓ | Final invariant (underestimate) |
| 366 | err.mulUpXpToNpU(mulDenominator) | ↑ | Scale error by denominator |
| 373-377 | Relative denominator error | ↑ net | Account for denominator error |
with in mind this idea:
| Output | Direction | How Achieved |
invariant | ↓ underestimate | sqrt↓ × mulDenom↓ × mulDown↓ = triple underestimate |
err | ↑ overestimate | divUpMag↑ + ×20 + mulUpXpToNp↑ + ×40 + final+1 |
How to build it ? Start from the output: decide who should "win" rounding for this function. Then trace backwards through each operation, marking whether it should round up or down to preserve that bias, any operation that rounds the "wrong" way gets flagged. The table is just: Line | Operation | Current Direction | Correct Direction | Match?


