[08/52] CAHSR April Finance and Audit

Original Video: https://www.youtube.com/watch?v=YNKlrY0_VOc

Introduction

Timestamp: ~0:00

At long last, the much-promised migration towards more traditional engineering things!

We're going to take a look at the April meeting of the Finance and Auditing Committee meeting for the California High-Speed Rail (or CAHSR) project.

In 2008, Proposition 1A passed in California, which authorized $8 billion to develop a high-speed rail system in the state. This is commonly used as the "starting line" for tracking project progress, although other efforts and studies had taken place for several decades prior.

We're going to take a look at this effort through the lens of program engineering for public projects. These efforts are very difficult! This finance and audit committee is a good way to keep track of how well the project is going and what controls (for cost and schedule) exist. Specifically, we want to think about this effort from an engineering perspective--how are things being built, how are they tracked & controlled, etc.

Project Overview

Timestamp: ~2:40

For context, here's a map of the overall effort. This is largely divided into "CPs", or "construction packages". CP1 is the middle segment going roughly from Bakersfield to Merced. Other CPs include the San Francisco and Los Angeles segments.

The first thing you'll notice it the parlimentary procedure. This is a great way for engineers to start thinking about how high-level committees and public events are organized. If you're not familiar with Robert's Rules of Order, you should check them out--it helps make a lot of these activities less opaque.

The project started 16 years ago, and is currently booked at about 4x (or just above $30 billion dollars) the original bond funds, so there are questions among taxpayers in the state: "where are our trains!?" This committee can help us understand what's going on and how success is being measured.

Accounting Records

Timestamp: ~4:22

On the topic of IG auditing, my understanding is that this part of the project has gotten much better compared to a somewhat messy past. Of course, this is still the state auditing the state!

The overall costs booked right now are somewhere a bit north of $30 billion. So, a lot of other funding sources are required. You'll see line items here for other funds like cap-and-trade balances, real property management funds, and disputes. There are a surprising number of disputes that exist for bills and charges between the state and the contractors working on the project. This comes about for several reasons, including changing requirements and differences in funding practices.

But a huge portion of the cash balance right now (north of 75%) are the cap-and-trade cash funds, which are largely based on credits sold and anticipated energy savings from ridership projections.

Another significant portion of funds, which they are enumerating now, comes from matching federal funds and other grants for large-scale projects and equipment purchases (like train sets).

But going back to the map you can imagine that the end segments that go through both mountains and densely populated urban areas. If they're still having trouble securing right-of-way and other property parcels for the flat farmland in CP1, you can only imagine how much more difficult that is going to become.

But more interesting are the specific project metrics being reported to track success:

• People working on the project

• Bridges completed (probably including viaducts)

• Utilities relocated

You can learn a lot in engineering by observing what metrics are being reported and "controlled" to track project success. For example, you can double the number of people working on the project but that doesn't necessarily increase the number of track laid in miles. Utilities are relatively low-hanging fruit to be reported, though they are getting better.

Bridges and viaducts, on the other hand, end up being a significant portion of the effort. While the line is going through flat farmland, a sizable portion of the line between Bakersfield and Merced is elevated above the land using viaducts, which is significantly more expensive.

Disputes

Timestamp: ~11:22

Disputes have been a significant obstacle for accounting over the past 9 months I've been following these committee meetings.

But 98-99% for one specific issue in disputes is... very high. This would merit risk mitigation as it likely risks cost/schedule controls.

One California-specific issue here is, the state budget is just big enough (many billions) that something like 8 digits of disputed funds are something of a rounding error. But for a project that's been running over 16 years, that can add up to significant accounting inconsistencies.

Other interesting metrics include "how many empty seats" exist (or vacancies). Given how much of the project is contracted out, it's likely these are mostly administrative roles, so they shouldn't have a significant impact on the technical execution unless there are specific milestones they are needed for on the critical path.

The capital outlay is also interesting to look at. The construction allocation makes sense--you'd expect 80% or so to be going towards physical engineering. But this breakdown also includes continued hundreds of millions for right-of-way purchases and third-party payments jumps out at me. Those are also a lot of legal fees enumerated on the following line!

Expenditures and Funds

Timestamp: ~16:12

Here's an interesting enumeration of specific federal grants and other funding sources. There are $5.5 billion from two specific grants: ARRA and the Federal State Partnership (FSP).

$11 billion dollars is a lot of "active" contracts, though, 16 years into a construction project. This is a lot of money being spent. The small business utilization is interesting as well--a significant portion goes to professional services, in part because a lot of the physical engineering will be done by medium-to-large firms that don't qualify for that construction line item.

This is also because there are specific small business requirements attached to bond. Very rarely do you find federal grants that don't have some sort of conditions attached. It's not just free money--states are required to satisfy specific requirements.

But... there's nearly $3 billion set aside in contingency for construction packages that haven't even started yet. This does jump out at me. This may be because they anticipate construction through highly-populated urban areas to have significant funding issues (a reasonable concern).

Another item that jumps out at me is the "other contingency" line items. Specifically, there are nine digits of funds allocated for unallocated contingencies--slightly nebulous. There are another 9 digits of funds allocated for "interim use", which is also nebulous. And then there's the "other" item, the largest in the trio, which indicates someone is having a hard time finding the imagination to categorize these contingencies in a consistent logical manner!

Preliminary May Report

Timestamp: ~21:43

This is a little strange to report outlays month-to-month. Year-over-year trends are much more useful and more likely to be seen for federal funds. After all, they are allocated by fiscal year so listing awards on a monthly basis doesn't make a lot of sense.

We also see more detailed enumeration of the vacancy rates. 69 positions could be a lot if the critical path is affected by 1 in every 7 co-workers are missing!

I also have to wonder how much of the funding stability is affected by continuing resolutions at the federal level. That can have significant impacts on what funds can consistenly carry over year-to-year, especially since grants are often a one-time program and not a "program of record".

One of the things you'll see, though, even in California, is that you have find money wherever you can. Each year racks up billions of dollars in additional expeneses (and don't forget inflation!), so if the federal government offers you money for (for example) ensuring sustainable electric powering of a train station, you make the adjustments you need to qualify for that funding.

One very illuminating resource I would recommend is the classic text "The Power Broker", a look at the life of Robert Moses. Moses ruled the civic engineering landscape in New York City, and there are a variety of passages that describe how he managed to piece together funding for massive projects that the city would never be able to afford. A few billion from Albany here, a few billion from DC there, and eventually he was able to build these massive bridges and parkways. And not entirely through admirable means, I should mention.

Finance vs. Auditing

Timestamp: ~27:08

I do think it's interesting that they have a single committee titled "Finance" AND "Auditing". Typically, of course, auditing is something you do to the finances. They did mention the state IG did some auditing work, but it's still the state auditing the state--a little bit like the fox auditing the henhouse.

The first thing we notice for this central valley report is that CP2 and CP3 are responsible for the majority of expenditures. Not only is this a lot--and may indicate CP1 construction may actually be winding down--it's a lot for segments that aren't actually undergoing engineering efforts yet at the terminals of the line.

Construction progress is a little difficult to track when they don't show consistent month-to-month (or quarter-to-quarter) progress. Are these funding issues? Changing scope and requirements issues?

Utilities can be a lot of things. It's one thing to restring overhead wires for power, but large sewage pipes are more difficult. But the real outlier here are telecoms, because they are regularly modernized. For a project that started in 2008, there's probably a lot more that could have been done upfront to ensure that lines (currently only 5-10 years old, in all likelihood) weren't being laid down across a known future rail grade.

It's also interesting to note how parcels are organized for acquisitions. Are rail parcels for actual grades? Maintenance stations? And there's a lot of questions and risk involved in the environmental clearance schedule (especially in California, where these regularly siderail large engineering projects).

Design and Build

Timestamp: ~32:01

I do wonder why they have to bundle design and build. Typically in software or space/defense, for example, you have separate stages for design; development; and deployment or operations/sustainment. At this point, I would expect this to be mostly "build", not so much "design", so a breakout would be interesting. Of course, a lot of this is outsourced in packages to different contracts so if they're not organized that way at the top level it will be difficult to "flow down" the same organization for reporting.

If I was chief engineer in this project, I would probably (instead of labor rates) want to see metrics expressed in terms of miles: how many miles are graded; how many are railed; and how many are ready for train sets (powered, instrumented, etc.).

The utility relocation is interesting because it is now starting with the previous month. This is a little different from previous reports, which show year-over-year progress, which does jump out at me because it makes it harder to consistently track long-term progress between committee reports.

The enumeration for utilities is even more interesting, because we see specific percentages broken down across CPs. The high number of electric utilities "NOT STARTED" in particular is a big surprise.

Property and Right-of-Way

Timestamp: ~34:42

Right of way is a BIG issue. And this report is very impressive, given that they report CP2-3 and CP3 in the same diagram. I don't know if those numbers are going to stay consistent, though, given that they're still approving environmental impact and soliciting community feedback. Those plans will change, and other property acquisitions will be needed.

The "earned value" chart is fundamental "bread and butter" for these efforts. It's a good tool, tracking "burndown" (towards 0, or progress up to 100%) of time and resources. It's a useful way to compare against risk (+/- delays, for example) and evaluate specific "controls" to bring rates back to solid projections. (I'm kind of surprised at least half the deck here isn't comprised of more charts like this.)

And here's the "design build" question! It sounds like they've realized the same concerns. "Design-bid-build" probably means you have different performers and contractors bidding on specific work segments, which is promising (if potentially more complicated and time-consuming). In particular, as mentioned above, this could help a lot with better reporting breakdowns when comparing design vs. construction tracks for engineering efforts.

What Have We Learned?

Timestamp: ~38:15

Obviously, this is a complex effort. A big part of the challenge is that the goalposts are moving, the fact that the bond funds have mostly been spent, and the fact that you need to patch together funding from wherever you can find it (which isn't unusual for large-scale civic engineering projects).

But fundamentally, there are two ways to approach complex efforts like this:

1. You can start with the easy part, hoping to iterate on lessons learned and build momentum as you show progress. The problem with this approach, as it has been taken by CAHSR, is that spending all of your money to build a connection between small cities in the central valley, is that it becomes difficult to make reliable ridership projections.

2. You can start with the hardest part. If you can figure out the hardest part, everything else should fall into place and be relatively easy. One of my favorite examples is the work of Hyman Rickover. We had nuclear reactors on submarines before we had them on aircraft carriers! But because we started with submarines (a tightly-confined space where nothing can go wrong) it then became much easier to adapt that technology to massive aircraft carriers and even other large surface vessels like heavy cruisers.

In this case, it would make a lot more sense to start with the "hardest" part by connecting large cities at the ends (in the north, connecting San Fransico with San Jose and Sacramento; in the south, connecting Los Angeles with San Diego and Riverside or Anaheim). Not only do you push through the densely-populated urban construction difficulties first, you would be able to establish more profitable ridership numbers for efforts that could start funding follow-on efforts across the state.

Florida has been able to put together an effort, more privatized than CAHSR, called "Brightline" that (while traditional diesel-electric) has passenger rail up to 100 MPH. They are raising funds (public and private) to also build out a "Brightline West" connecting Las Vegas to locations around San Bernardino. The approaches are very different and (while not entirely apples-to-apples, since they are building across the Mojave instead of occupied farmland) it will be interesting to compare.