Key insights
- A stock analysis app developer details significant bugs found in their engine, including incorrect Fibonacci levels, mis-scored unprofitable companies, inverted Monte Carlo scenarios, and CAGR calculation errors. While specific tickers aren't mentioned, the issues highlight broader risks in automated financial analysis and the importance of rigorous testing, especially with edge cases. This indirectly impacts US equities by underscoring potential unreliability in similar tools used by investors, creating a slightly negative sentiment.
We've been running a stock analysis app for a few months and recently did a deep audit of the analysis engine. Found some embarrassing bugs and fixed them all. Sharing for transparency and in case anyone building similar tools finds it useful.
The worst ones:
1. Fibonacci levels were mathematically wrong The support/resistance levels were calculated with two different formulas that happened to produce the same number. Result: the same price level appeared in both supports AND resistances simultaneously. Classic copy-paste error that went unnoticed because the output "looked reasonable."
2. Unprofitable companies scored as "Excellent" A company with negative ROE and negative margins was getting a health score of 81/100. Fixed with explicit handling: negative ROE → -15pts, negative margin → -10pts.
3. Long-term Monte Carlo scenarios were inverted The "Base" scenario was showing better returns than the "Optimistic" scenario. This was caused by historical drift being used instead of a fundamental-based drift, causing percentile ranges to cluster incorrectly.
4. 2-year CAGR was using a 3-year lookback We were using iloc[0] (oldest data point, ~3 years back) with an exponent of 1/2 (assuming 2 years). Fixed to iloc[-504] (504 trading days ≈ 2 years from the end).
Other fixes:
- PE ratio negative values showing as "cheap" valuation * "Approaching Death Cross" treated the same as confirmed Death Cross in recommendations * Analyst price targets returning 6 decimal places ($892.119284 instead of $892.12) * ROE >50% now shows real value with context note: "possibly amplified by share buybacks" * Strengths/weaknesses limit raised from 5 to 7
Lesson learned: Always test with edge cases, unprofitable companies, stocks at 52-week highs/lows, and non-US tickers. Most bugs only showed up when we manually compared outputs across multiple real tickers.
If you want can check on: -stocksanalyzer.app-