Unlock Donor Insights: Itemized Donation Analysis

Hey everyone! Let's dive into a critical enhancement for campaign finance transparency: itemized donor concentration analysis. Currently, our system treats all itemized donations (those over $200) the same. But this approach hides significant differences in how campaigns are funded. Imagine two candidates with identical itemized donation percentages – say, around 41%. On the surface, they appear similar. However, one candidate might receive that 41% from thousands of middle-class donors giving smaller amounts, while the other gets it from a few wealthy individuals contributing much larger sums. This disparity matters because it reveals who is truly funding our representatives. Are they supported by a broad base of individuals, or are they reliant on a small group of elite donors? Understanding this difference is key to promoting transparency and accountability in our political system.

The Problem: Hidden Funding Models

Currently, we're operating with limited information. We have aggregate totals from the FEC (Federal Election Commission), such as total itemized donations and total grassroots donations. While this data is helpful, it doesn't tell the whole story. We're missing crucial details about the number of unique donors and the concentration of donations. For example, we can't easily determine what percentage of a candidate's itemized funding comes from their top 10 donors. This lack of visibility can be misleading. Some candidates might appear to have widespread individual support, but in reality, they're primarily funded by a small, wealthy elite. The data to uncover these hidden funding models exists within the FEC Schedule A, which details individual transactions. The challenge is accessing and analyzing this data in a meaningful way. By extracting and analyzing this data, we can shed light on the true sources of campaign funding and provide voters with a more accurate picture of who is influencing our political process. This transparency is essential for a healthy democracy, allowing citizens to make informed decisions about the candidates they support.

The Solution: A Dedicated Analysis Worker

To address this challenge, I propose creating a dedicated Cloudflare Worker called taskforce-purple-itemized. This worker would be responsible for fetching, processing, and storing itemized donation data from the FEC. Think of it as a specialized tool designed to uncover the details behind those aggregate numbers. This worker will run on a scheduled cron, meaning it will automatically execute at regular intervals – ideally every 1-5 minutes. During each run, it will process data for a small number of candidates, staying within the free tier limits of Cloudflare Workers. This approach ensures we can analyze a large dataset without incurring significant costs. The worker's primary task is to fetch the full Schedule A transaction data from the FEC API. This data contains detailed information about each individual donation, including the donor's name, the amount of the donation, the date of the transaction, and even employer information. Once the data is fetched, it will be stored in a dedicated KV namespace, which is essentially a key-value store optimized for fast data retrieval. This allows us to access and analyze the data efficiently whenever we need it.

Unlocking Insights from Schedule A Data

Imagine the wealth of information we unlock once we have all the itemized transactions at our fingertips! We gain access to: Every individual contributor's name, allowing us to deduplicate entries and calculate the unique donor count; every transaction amount, enabling us to analyze the distribution and concentration of donations; dates, locations, and employer information, which can be used for future analysis; top donor concentration, revealing the percentage of total itemized donations from the top 10 donors; the Gini coefficient, a statistical measure of donation inequality; repeat donor patterns, distinguishing between one-time and sustained support; and the average donation size, serving as a proxy for wealth concentration. The most important benefit is that once the raw data is stored, we can perform ANY analysis without repeatedly fetching information from the FEC. This future-proofs our system, allowing us to adapt to new analytical needs and refine our understanding of campaign finance dynamics.

Staying Within the Free Tier

Now, let's talk about resource requirements. We need to ensure our solution remains within the free tier limits of Cloudflare. FEC API calls are estimated at around 10,700 total, which is manageable with an enhanced rate limit obtained via email. Cloudflare Worker CPU usage should be compliant, as we're allocating only 10ms per request and processing 1-2 members per run. KV storage is estimated at around 535 MB for raw transactions, fitting comfortably within the 1 GB free tier limit. KV writes are also well within the limit, as we'll be processing approximately 535 members, resulting in 535 writes, which is far below the 1,000 writes per day limit. The initial data population will take an estimated 4-22 hours via the scheduled cron. This is because we're breaking down one giant, CPU-intensive job into hundreds of tiny jobs, each fitting within the free tier limits. This clever approach allows us to achieve our goals without incurring any costs.

Storage Strategies: Raw Data vs. Aggregated Stats

We have two main options for storing the data: storing raw transaction data or storing only aggregated statistics. I recommend storing raw transaction data. This involves storing every transaction with details like contributor name, amount, date, employer, and more. This option requires approximately 535 MB of storage, which is 53.5% of the free KV tier. The major benefit is that we can calculate ANY metric in the future without re-fetching data from the FEC. This future-proofs our system and allows for maximum flexibility. The alternative is to store only aggregated stats, such as unique donors, average donation, and top donor percentage. This option is more space-efficient, requiring only about 2.7 MB of storage. However, it locks us into these specific metrics forever. If we need to perform a new analysis, we'll have to re-fetch the data from the FEC. Given that we have ample space within the free tier, storing raw data is the clear choice. It provides us with the flexibility to adapt to future analytical needs and ensures we can always access the most granular level of information.

KV Structure: Organizing the Data

To keep everything organized, we'll use a specific KV structure within the ITEMIZED_ANALYSIS namespace. We'll have keys for transactions:{bioguideId}, which will store an array of all itemized transactions for a given candidate. We'll also have keys for stats:{bioguideId}, which will store pre-calculated concentration metrics for that candidate. Additionally, we'll maintain a processing_queue to track which members still need analysis and a last_processed key to monitor the progress of the scheduled cron. Each transaction record will contain fields like contributor_name, amount, date, employer, occupation, city, and state. The stats record will include fields like bioguideId, totalItemized, transactionCount, uniqueDonors, avgDonation, medianDonation, top10DonorPercent, top100DonorPercent, giniCoefficient, and lastUpdated. This structured approach ensures we can easily access and analyze the data we need, when we need it.

Implementation Roadmap

Let's outline the steps to bring this solution to life. First, we'll create the new Cloudflare Worker (taskforce-purple-itemized) and configure it with a scheduled cron trigger that runs every 1-5 minutes. The worker will process 1-2 members per invocation, fetching all Schedule A transactions in a paginated manner and storing them in KV. Next, we'll create the KV namespace (ITEMIZED_ANALYSIS), keeping it separate from the main member data and dedicated to transaction storage. We'll then update the main worker to read the itemized stats from the ITEMIZED_ANALYSIS namespace, joining them with the member data for tier calculation and incorporating concentration metrics into tier penalties. For the initial data population, we'll queue all 535 members and process them via the scheduled cron, tracking progress in the processing_queue. Finally, for ongoing updates, we'll re-fetch data quarterly to match the threshold recalculation schedule, prioritizing members with recent activity. This phased approach allows us to build and deploy the solution incrementally, ensuring stability and minimizing disruption.

Concentration Metrics: Impacting Tier Calculations

So, how will these concentration metrics impact tier calculations? We'll introduce a Concentration Score that penalizes candidates with highly concentrated funding. For example, if the top 10 donors contribute more than 25% of itemized donations, a high concentration penalty will be applied. Similarly, if the top 100 donors contribute more than 60%, a medium concentration penalty will be applied. We'll also penalize candidates with a small donor base (less than 500 unique donors) or a high average donation size (over $2,000), indicating wealth concentration. This new system will differentiate between candidates with similar itemized donation percentages but vastly different funding models. A member with 41% itemized donations from 2,000 wealthy donors will likely see their tier drop due to the concentration penalties, while a member with 41% itemized donations from 13,000 middle-class donors will likely maintain their tier, reflecting their broad base of support. This reveals the actual funding model with greater transparency.

Key Benefits of this Enhancement

This enhancement offers several key benefits. First and foremost, it provides greater transparency, allowing us to see who is truly funding our representatives. It also improves accuracy by distinguishing between concentrated and broad individual support. The future-proof nature of storing raw data enables any future analysis without re-fetching data. The solution is free tier compliant, ensuring we can operate within the limits of Cloudflare's free tier. It's also scalable, making it easy to add new metrics without re-fetching data. And finally, it's defensible, relying on hard FEC data rather than estimates or intuition.

Next Steps: Making it Happen

Here's what we need to do next. First, we need to get an enhanced FEC API rate limit by sending an email request (it's free!). Then, we'll create the taskforce-purple-itemized worker repository and set up the ITEMIZED_ANALYSIS KV namespace. We'll implement the scheduled cron processor and test it with a small batch (10-20 members). Finally, we'll deploy and populate all 535 members and update the tier calculation with concentration penalties.

This is the missing piece for achieving true campaign finance transparency. The data is out there—we just need to grab it!