John Yang is a financial engineering student at Columbia Engineering with experience in quantitative modeling, markets, and applied analytics. He is currently working with Professor Michael Robbins on a synthetic data generation project for PWL, focused on building more realistic market scenarios for financial planning and risk analysis. His work examines how advanced simulation methods can better capture asset behavior, tail risk, and periods of market stress than traditional models.
In this episode, Ben Felix and Braden Warwick unpack the surprisingly complex world of expected return modeling and why it matters so much for retirement projections, portfolio construction, and financial advice. They explain how PWL Capital currently estimates expected returns across asset classes, why traditional Monte Carlo methods relying on Gaussian distributions may miss important market behaviors, and how new research could improve the realism of long-term financial planning simulations.
The conversation also explores a fascinating collaboration between PWL and Columbia Engineering student John Yang, who worked with Professor Michael Robbins on a project to build more realistic synthetic return data for financial planning. John explains how his team used empirical distributions, t-copulas, and Extreme Value Theory to better capture market crashes, fat tails, and asset co-movements during periods of stress. Ben and Braden then analyze how these improved simulation methods affect financial planning outcomes, sustainable spending estimates, and projections for long-term wealth accumulation.
Key Points From This Episode:
(0:00:04) Introduction to expected return modeling and why it matters for financial planning.
(0:00:25) The importance of volatility, correlations, distribution shape, and time-series behavior in portfolio projections.
(0:01:26) How Scott Cederburg’s research on block bootstrapping influenced PWL’s thinking on simulations.
(0:02:03) Introduction to Columbia Engineering student John Yang and the industry research collaboration.
(0:03:30) How Conquest Planning allows PWL to upload custom return simulations.
(0:04:05) A new PWL client’s detailed reasoning for moving from DIY investing to working with an advisor.
(0:06:22) Why financial planning and Monte Carlo simulations were central to the client’s decision.
(0:07:22) Cross-border financial complexity and the value of professional advice.
(0:08:03) Estate planning, cognitive decline, and the role of trusted financial relationships.
(0:10:02) Research on cognitive decline and its impact on financial decision-making.
(0:12:00) Delegation, accountability, and reducing mental overhead through advisory relationships.
(0:13:47) Why the client chose PWL specifically and the appeal of evidence-based investing.
(0:15:25) Ben and Braden discuss the perceived disconnect between online discourse and demand for AUM advisors.
(0:16:12) Overview of PWL’s methodology for estimating expected returns across asset classes.
(0:17:05) How PWL combines historical returns with market-implied expected returns.
(0:18:07) The use of factor premiums and expected return composition in taxable projections.
(0:18:48) Why PWL previously relied on Gaussian multivariate normal distributions for simulations.
(0:19:41) Arithmetic vs. geometric mean returns and why the distinction matters.
(0:21:01) A simple example illustrating volatility drag.
(0:23:29) Why diversification benefits must be incorporated into expected portfolio returns.
(0:25:15) How correcting portfolio math improved expected return estimates by 20–30 basis points.
(0:27:12) Transition to John Yang’s interview and introduction to synthetic data generation.
(0:30:07) John explains the limitations of Gaussian return assumptions.
(0:31:04) Why realistic sequences of returns matter for retirement planning.
(0:32:16) Empirical evidence that returns are not truly random.
(0:33:25) The three modeling challenges: unique asset behavior, realistic co-movement, and tail risk.
(0:37:49) Separating marginal distributions from dependency structures in the modeling process.
(0:38:48) Using a t-copula to better model asset co-movement during market stress.
(0:39:39) Why historical data alone struggles to capture rare crisis events.
(0:40:06) Applying Extreme Value Theory and Generalized Pareto Distributions to model tail risk.
(0:42:15) How Monte Carlo simulations generate many realistic future return paths.
(0:43:00) Imposing forward-looking expected returns and volatility assumptions onto the simulations.
(0:44:56) How the new framework better preserves skewness and kurtosis.
(0:46:38) Evaluating the new model using marginal shape, tail behavior, and co-movement scores.
(0:48:10) Why the new model significantly improved tail realism without sacrificing correlations.
(0:49:05) Future extensions including dynamic correlations and volatility clustering.
(0:50:28) Potential future use of GANs and machine learning for synthetic financial data.
(0:52:02) Key takeaway: financial planning requires realistic return paths, not just summary statistics.
(0:53:41) Braden analyzes how the new simulation framework affects financial advice.
(0:55:04) Why monthly index data produced fatter tails than long-term annual DMS data.
(0:58:47) The new model improved Monte Carlo success rates by roughly 2–3%.
(1:00:25) Sustainable spending estimates changed only modestly under the new simulations.
(1:02:27) Why the improved methodology matters more for alternative asset classes.
(1:04:25) The surprising finding that median wealth outcomes increased while mean outcomes decreased.
(1:05:47) Why Gaussian simulations can create unrealistic runaway wealth scenarios.
(1:07:20) The practical implications for estate planning and multi-generational wealth projections.
(1:08:30) Why better simulation methods are especially important for concentrated and alternative investments.
Read The Transcript:
Ben Felix: This is the Rational Reminder Podcast, a weekly reality check on sensible investing and financial decision-making from two Canadians. We are hosted by me, Benjamin Felix, Chief Investment Officer, and Braden Warwick, Financial Planning Product Architect at PWL Capital.
Braden Warwick: Welcome to episode 411. Today we have a pretty nerdy, but very interesting episode on expected returns.
Ben Felix: It is really interesting. So it's like figuring out what to use as an expected return for stocks and bonds. Like what is the average return that you expect or any other asset class is super important.
It's one of the most important parts of financial planning and portfolio management. It ultimately informs how you allocate your assets, how much you need to save or spend in your financial plan. Higher expected returns make an asset class more attractive, all else equal, but of course all else is not always equal.
Higher volatility makes an asset less attractive, all else equal, but again, all else is not always equal. And lower cross asset correlations make assets more attractive together in a portfolio, all else equal, but there are always trade-offs. It's never actually all else equal.
Modeling those relationships is hard enough, but there are other aspects of expected returns that get less attention while being similarly important. The big one there is the shape of the distribution. And another one is the time series characteristics of asset class returns.
That's like volatility clustering in stocks where a little bit of volatility tends to be followed by a lot of volatility, mean reversion in stocks, mean aversion in bonds. Those are all characteristics that can materially change optimal asset allocations and modeling outcomes like long-term financial planning outcomes. Now listeners will be familiar with Scott Cederburg's work on this, which we found illuminating.
He and his co-authors use block bootstrap to preserve the time series characteristics of historical stock and bond and cash returns to show that doing that changes the relative attractiveness of stocks and nominal bonds and cash for long-term investors. Since hearing from Scott and learning about his research and reading his paper, we've been trying to figure out, basically me and you, Braden, have been trying to figure out how we can improve our expected returns modeling when we're running financial planning projections. That's what we're going to talk about in this episode.
Braden, you're going to talk about some of the modeling considerations that go into expected returns modeling and what we're doing now. This part's pretty cool. We're going to hear from John Yang, who's a financial engineering student at Columbia Engineering, the engineering school at Columbia University.
PWL engaged with John and some of his classmates for their industry project where the students supported by their professor, Professor Michael Robbins, aim to solve a real problem for a firm. It's pretty cool. There are a bunch of industry projects that are available to students and they get to choose which one they want to do.
I talked to Professor Michael Robbins, explained this expected returns modeling challenge that we had, and he was like, cool, I'll propose that as a project. John and his group of students for the class chose our project and worked through it for a semester. It was a pretty cool experience. Hey, Braden, that was cool.
Braden Warwick: It's stuff that I did when I was in academia, working with industry partners. I always found that pretty impactful when you're actually able to make a difference on real product and real tangible research that can be applied in the real world. I think hopefully they took that same experience in working with us, but it was definitely really cool to be on the other side of that where we were able to leverage academia and feel like we're in the bleeding edge of this type of research. Yeah, really cool experience all around.
Ben Felix: Yeah, I thought it was cool. Our financial planning software that PWL uses right now is called Conquest Planning. They're just Conquest.
The way that its returns analysis works, when you want to look at how sustainable is this financial planning projection over a projected range of future outcomes, we upload a thousand runs of simulated data. Doing it that way is really cool because some other financial planning softwares will run a Monte Carlo inside the software, but you don't get to change any of the parameters or the shape of the distribution, but because Conquest lets us, or the only way that you can do it is by uploading your thousand runs of data, you can use whatever simulation method that you want and then upload that into the software. Because of that, we are able to do stuff like what John and his team worked on, where we define our own way of creating a distribution of returns, and then we can test that in financial planning software. We don't yet have this loaded up in Conquest, but Braden has done some modeling work just to look at how John's method of simulating returns compares to what PWL had been doing previously in a long-term projection.
How does it change the outcome in these projections? We will get to that in a minute. That's the setup.
Now, I do want to speak real quickly about something cool that happened before we just get into the main content. A new PWL client, who is characteristically, as many of them are, highly analytical and had previously been a successful DIY investor, they wrote down a detailed analysis of their decision to become a PWL client. I believe that they wrote this to share in a group chat with some of their friends.
The cool thing is they also shared the analysis with their new PWL advisor, who then shared it with me, and then we got permission to talk about it on the podcast because I thought it was pretty thoughtful analysis. I do have to make some quick disclosures before we get into this. The person who wrote this is a client of PWL Capital, which is a subsidiary of OneDigital, who also operates OneDigital Investment Advisors, LLC, in the US.
As some of our audience is US-based, we wanted to properly disclose that the client was not compensated by PWL Capital or OneDigital for writing this. Also, keep in mind that clients' experiences vary, and therefore, their views are not representative of all client experiences. Additionally, the testimonial is not an indication of past or future investment performance.
All right, nice disclosure there to make Ross happy. I'm going to get into what this client wrote. Keep in mind, this is a long post that they wrote in a group chat type setting.
"Here's what factored into my decision to make the change from DIY couch potato style investing to an assets under management financial advisor. This is just my own thought process. I'm not trying to change anyone's opinion here, but hopefully the share is interesting.
I've been self-directed my entire life and have spent the past 10 plus years getting serious about building up more in-depth knowledge about the pros and cons of different approaches to investing. I don't have particularly complex needs, and I've been a long VEQT for ages now, so it took quite a while to work through my hesitations to the AUM, the assets under management model. Even a few months ago, I was pretty sure I wanted a fee only advisor.
This is pretty fresh and not informed by actual experience yet. I'll start with why I went with an advisor at all, roughly ordered from most important to me to least. Financial planning, my top reason for finding an advisor.
I'm starting to think about how to optimize for funding my life after I stop working. I've built fairly complex spreadsheets to project out drawdowns across my accounts to get a baseline idea of where I'm at. Even with tax planning included, they're still crude and unoptimized compared to something like Conquest, which again is the software that PWL uses.
An advisor can use it to model and optimize drawdown scenarios, then run thousands of Monte Carlo market return simulations." That's actually a good tie-in because that's what we're talking about. "To help buffer against unexpected conditions that build robustness and security into the plan.
I knew that I couldn't effectively plan for the future without this level of specific planning capability. That was totally validated the first time we ran our numbers through Conquest. I was way off, but in a pleasant surprise kind of way.
The next point is cross-border complexity. One spouse in this couple is an American citizen, which has prevented us from combining our finances beyond our mortgage. We'll miss out on long-term planning strategies, for example, spousal RRSPs, without qualified guidance on how we should be approaching this.
The spouse is looking for more peace of mind from qualified advice on dealing with the IRS, despite having separate accountants to deal with taxes in each country. There have been some unexpected tax bills due to filing issues. The next point is estate security.
We don't have kids, but my wife and I are in different financial positions from both a knowledge standpoint as well as account balances. Odds are I'll die first. Establishing a relationship now with an advisor we both trust who can be relied on to look out for both spouses' best interests in the event that one of them is gone is a huge benefit to everyone's peace of mind.
The next point is a hedge against decline." They write, "my father-in-law is in the early stages of dementia and we're already seeing how that's having an impact on his finances. Having an outside observer who can spot when conditions change and help intervene before costly mistakes are made is great insurance."
I did want to add a note here. PWL always encourages clients to assign a trusted contact person, which is someone that the client trusts who PWL is pre-authorized to contact if something doesn't seem quite right. If someone wants to make a trade or move money to a different country or something like that, if the PWL advisor is like, I don't know if this, something doesn't seem right here, they can contact the trusted contact person who can then speak with a client or confirm whether everything's okay or whatever the case may be.
There is a paper that found that older people tend to underestimate their own cognitive decline and that those who have experienced a severe cognitive decline, but are unaware of it are more likely to suffer wealth losses compared to those who are aware or did not experience a severe decline. This is a 2024 study. Are Older People Aware of Their Cognitive Decline? Misperception and Financial Decision-Making?
The paper examines data from the Health and Retirement Study, a representative panel of the US population age 50 plus to study the relationships between self ratings of memory changes, assessed changes in memory performance and wealth changes across waves of the survey. Results are pretty crazy. They suggest that there is a causal role of unawareness of cognitive decline for wealth losses and that wealth losses among unaware respondents mainly reflect a decrease in the value of their riskier assets, their investments, their stock portfolio.
The authors also find that wealth losses are concentrated in the highest wealth quartiles. Pretty crazy data. Very relevant study to that point about a hedge against decline.
The authors of the paper interpret their findings as the effects of overconfidence where people who have experienced cognitive decline but do not realize it end up making poor financial decisions that can be very costly. Now we're back to this person's post explaining their decision to hire PWL. "The next point is core competency.
Having industry experts stay on top of emerging financial research, tax code updates, insurance products and other changes that might be relevant to me means less things I need to do myself. Sure, I have the basics covered but beyond that, is there a new tax credit I should be taking advantage of? Do I have the right set of insurance products at the right levels of coverage?
Is there a new financial research that indicates I should be considering a change in my strategy? Does my will offer the asset protection I think it does? It could be a full-time job just trying to stay on top of everything that could be relevant to me.
A whole team who does this for a living and will get in touch when new information might apply to my situation sounds better than trying to do it myself poorly. The next point is dealing with life events extending that last point when big life events happen, particularly unexpected ones. I also have a trusted advisor who can help me make good decisions, particularly in stressful times.
Do I know I'm getting good quotes the next time I renew the mortgage? How do my wife and I best handle the inheritance windfalls coming our way in the next 10 to 15 years if one of us dies prematurely? How does the other plan for the rest of their financial life?
The next point is objective third-party advice. I'm pretty sure I've made every investing mistake you can make in the past, so I feel fairly confident I won't make those ones again, but I'll make new ones. I'm not immune to doubt as market conditions change.
I remember 2008, but I had such a small amount invested back then. I can't call how I'll react the next drop like that. Having a steady hand there to keep me on course isn't the first benefit I think of, but I've worked with a personal trainer in the past and saw pretty directly how having someone for accountability can keep me on track, so I trust that will apply here too."
That is an interesting one. I think I mentioned on the podcast before. I have been working with a personal trainer for accountability and gut checks on stuff.
I have found it to be pretty useful. One less thing to think about. "The next point here is freeing up mental overhead.
I actually didn't consider this at all before we signed on." This is an interesting one because they hadn't considered it before and then experienced it once they started working through onboarding with PWL. "Even through just getting started with PWL, I'm already noticing that the mental space I've been giving to long-term planning and building my financial knowledge is easing up.
Good example, I really, really wanted to discuss account drawdown optimization strategies late last year. Now I don't care, and that can wait until I'm ready to actually start doing it. There are lots of things I'd like to be good at, but only so many hours each day.
Removing this from the list makes room for something else." Good classic case of delegation. Then they had some notes on why they chose PWL in particular.
"The first point is high trust. I'm a Rational Reminder listener, so hello listener, if you're listening. I'm very familiar with the company and their approach.
I like their evidence-based philosophy and that they look at the investment strategy as only the tip of the iceberg of the services they offer. There are a lot of bad advisors out there who will happily take your 2% and provide very little value in return, but I have a high level of confidence that PWL is not that. Their fee is comparatively reasonable and they'll work hard for me to earn it.
It's already been noted in the thread." I guess this was elsewhere in the discussion thread that this was posted in, but "they're also the first to admit that if your needs are simple and you're a perfectly happy DIY investor using globally diversified index funds, you may not be a good fit for their services." I respect their pragmatism.
We have always said that. PWL doesn't make advisors in general and PWL don't make sense for everyone. Lots of people can be successful DIY investors, but this person's points on why they chose to work with PWL are pretty interesting.
"Then the last point is access to Dimensional, more of a nice to have, but moving from VEQT to PWL plus DFA takes some of the sting out of the overall fee increase. Plus I like factor investing and Dimensional doesn't offer ETFs in Canada, so the only way to access their products is through an advisor." This was pre-Avantis ETF, so maybe that's changed a little bit, but notably this is the very last point in their list.
They conclude, so yeah, "very little of the decision was influenced by anything about managing the investments themselves and more about holistic financial planning and services. It won't be for everyone, but the fit was there for our needs." The end.
I thought it was just kind of cool to hear from someone who had recently made the decision to work with PWL and then documented their own thought process. I thought it was worth sharing with listeners of the podcast.
Braden Warwick: I also think it's pretty cool that they went through the full range of investor profiles, if you will. They started as a DIY investor. They were obviously very interested in that, enough so to listen to our podcast on a regular basis.
Then they also were considering fee-only financial planning. I think that's a really popular service right now. There's a lot of talk around fee-only financial planning and a lot of people promoting it.
I don't want to discredit any of the good work that they're doing, but I also think that this client's really laid out the benefits of the AUM model and working with someone and working with a brand that they can trust over time. This spans just beyond working with one individual, that they're really working with a team of people that have that person's best interests in mind. I think it really speaks to what we're doing here. I thought it was really cool to read.
Ben Felix: That's probably why I found it so interesting. If you look online, everyone says, never go to an AUM advisor. You have to get a fee-only financial planner.
Meanwhile, PWL continues to grow considerably. We're adding lots of new clients that want to work with us. There's a disconnect somewhere there.
I think this person really filled in a lot of the gaps about why someone might choose specifically the AUM model, even when they know that the fee-only financial planning model is available. It's really that just delegation, peace of mind, stuff that's hard to measure combined with all of the expertise in financial planning and portfolio management services. Okay.
Into the main content here. Hopefully, people found that interesting and didn't come across as a sales pitch or something, but I thought it was worth sharing. I'm going to just talk a little bit about PWL's methodology for expected returns for estimating moments of the expected return distribution.
Then, Braden, you're going to talk more about a whole bunch of super nerdy stuff, including how we do our simulations now to generate those distributions. At a high level, PWL combines what we call a market-based expected return, which is an expected return that is implied by market prices. For stocks, we use the inverse of the Shiller-CAPE ratio, for bonds
we use the bond yield. That's one piece, the market-based expected return. Then, we use what we call the equilibrium cost of capital, which is the very long-term return for that asset class.
For that, we use the Dimson-Marsh-Staunton data that goes back to 1900. Then, we combine them in different ways, depending on the asset class. For equities, we use 75% historical and 25% market-based expected return.
That's 75% world equity return from the Dimson-Marsh-Staunton data. We do adjust that for valuation changes. If equity returns have been high recently because valuations have increased a lot, or if valuations have increased a lot and that's contributed to recent stock returns, we do adjust for that in that long-term measure.
Likewise, if stock valuations plummeted all of a sudden and the historical return dropped, we would adjust for that valuation change. Then, we combine that with 25% market-based expected return, which, as I mentioned, is the inverse of the Shiller-CAPE. It's not statistically robust, but we ran a bunch of regressions on how predictive has the Shiller-CAPE been historically for future returns.
A statistician would probably mock us for our methodology on this specific part, but we figured that a 25% contribution from the market-based expected return was roughly in line-ish, approximately, with the historical predictive power of that metric. Then, for fixed income, we do the inverse, where it's 25% historical return and 75% the current yield to maturity on bonds. Again, that's because historically, bond yields have been much more predictive of future realized returns than the CAPE has over future realized stock returns.
Again, they're not perfect statistically derived numbers. They're rough estimates that work well for doing our expected return estimates. Then, for factor tilted portfolios, we do add factor premiums to market cap expected returns.
We use historical factor premiums that we then shrink down for post-publication shrinkage of each factor. We also have to do estimates for the composition of expected returns. If you're doing a projection, particularly for a taxable investor, you need to split out how much of the return is coming from Canadian dividends, how much is foreign dividends, how much is interest, how much is realized versus unrealized capital gains.
We have methodologies for all of those things and include them in our data. Then, you have to estimate a correlation matrix. We use historical correlations for that.
We're currently doing our projections, our distribution of returns using a Gaussian distribution. We simulate returns using a multivariate normal distribution. That gives us our 1,000 runs of data that we can use for our planning projections.
That specific piece, that simulation process is the thing that we were working with John to try and improve. Braden, do you want to jump into the distinction between geometric and arithmetic mean returns and how it relates to these projections?
Braden Warwick: For sure. Before I get there, though, I just want to set the stage a little bit with regards to the research project and with what you'll hear in the interview with John, the problem that we gave him. We're using the multivariate normal distribution. We wanted to change the shape of the distribution, look at the higher order moments.
Where we ran into trouble in doing that was because we have so many asset classes. We have our market cap weighted asset classes, our factor tilted asset classes. Internally, we also have a bunch of other asset classes like alternative investments and individual stocks and things that we don't make public.
We also don't recommend to our clients necessarily, but it's really important for us to have in the planning software so that we can quantify those outcomes. If clients have questions about investing in a certain other type of asset class that we don't typically recommend, we can actually show how that impacts their financial plan directly rather than just having our story versus the story of another advisor that is trying to sell them something. So when we do that today, altogether, it ends up being, I think, 4 million data points that we end up sending over to Conquest Planning.
But it was going to be really difficult to add in those higher order moments in the way that we're doing it now. So that's why it was really an exciting opportunity to engage John and a dedicated research team led by Professor Robbins to look at the better ways of doing this. Coming back to geometric versus arithmetic means, I really want to set the stage here, too, because this is an important one that I think most people are aware of, but I'm not sure that everybody fully understands when to use what and the implications.
Before I get into the details, I think it's important just to backtrack and define what a geometric mean is and what an arithmetic mean is. I'll use a simple example. So that people can follow along.
If we have two years of returns, year one is a negative 10% return and year two is a positive 10% return, we can see that the simple average of those two numbers is 0%. So that is the arithmetic return. But if we walk through what happens to your investment account subject to those returns over time, if you have $100 and it has a negative 10% return in year one, it comes down to $90.
And then if it's subject to a plus 10% return, that $90 becomes $99. So at the end of that two-year period, you started with $100, you're left with $99. Even though the arithmetic mean is 0%, you're still down $1 and you have a negative realized return over that two-year period.
So that is the need for the geometric mean. From that example, it should be clear that if you're doing a multi-year projection, multi-year financial planning projection with a constant rate of return, you need to use the geometric mean. But if there's volatility included, if you're able to use a varying rate of return year over year, then you can use the arithmetic mean and sample from that distribution accordingly.
Ben Felix: And then the geometric mean of that distribution, well, would be the geometric mean, I guess. If you run a bunch of simulations using the arithmetic mean and the other moments of the distribution that you have, the geometric mean of those distributions will be same as the calculated.
Braden Warwick: They should match, exactly. So if you sample from that distribution of arithmetic means, and then you use that data in your financial planning projection, then the geometric mean will come out as the end result.
Ben Felix: If you're sitting down using Excel or whatever to do a projection and you're just using a constant rate of return, you want to use the geometric mean. But if you're sitting down and simulating a whole bunch of possible returns, you need to use the arithmetic mean and the other moments of the distribution.
Braden Warwick: That's right. If you have access to the higher-order moments of distribution, which we'll get to with our interview with John. But simply speaking, the difference between the geometric mean and the arithmetic mean is just the volatility drag.
That's what it's called. And it's nothing crazy. It's just half of the variance or half of the standard deviation squared.
You can easily calculate that based on the data that we publish in our expected returns paper. There's nothing really more to it than that. But it is important.
There's a really important implication when you're calculating the portfolio expected return, because that volatility drag of the portfolio is different than the volatility drag of those asset classes in isolation. We hear all the time that diversification is the only free lunch in investing. So I think it's really important that you include those diversification benefits in calculating the expected return of the portfolio.
Let me walk through how to do that. In our expected return paper, we publish the geometric return of the asset classes in isolation. And then we also publish the expected return of the portfolio.
Our paper has everything in geometric return. So it's simple for people to interpret and to do their own projections with if they want. But there's kind of this intermediate step.
You have to convert the asset class geometric means back to their arithmetic means. And then you can take the weighted average of the arithmetic means to come up with the expected arithmetic mean of the portfolio. But then in order to get the geometric mean of the portfolio, you need to know what the standard deviation of the portfolio is.
And in order to do that, you need the correlations of those asset classes along with the standard deviations of those asset classes and the weighting that they make up in the portfolio. So once you're able to do that, then you have the standard deviation of the portfolio and you can easily convert back to the geometric mean of the portfolio. You may think that this is just unnecessary.
It's just complex. It doesn't really add any value, but it provides, well, based on our current data, we see about a 20 to 30 basis point increase in the expected return of the portfolio, depending on the asset mix when we do it the proper way, which is converting to arithmetic means, combining the arithmetic means and then converting back to the geometric mean of the portfolio. And the difference, that 20 to 30 basis points is the benefits of diversification and the correlation benefits that you get from holding those different asset classes in a single portfolio.
Ben Felix: Yeah, it's really interesting. If you're not doing that, if you're not going through those steps to calculate the portfolio level expected return, you're underestimating your expected returns and you're ignoring the effects of the positive effects of diversification.
Braden Warwick: That's right. And 20 to 30 basis points that can exceed fees in some cases. So it is a meaningful number that I think should be included.
And it's worth noting too, that one, Conquest will do this automatically for you. It has the proper portfolio math baked in so that if you provide the asset class expected returns, that it will do that conversion properly. And two, previously we also were making that same mistake.
So we were not immune to this. We found the mistake, corrected it. And I think that also kind of speaks to the client testimonial that it's nice to have a whole team of people, a financial advisor to diagnose that issue and fix it as a bit of a tall task because of all the other work that they have going on.
But having Ben and I and other people to think through this stuff and to take care of it in Conquest, I think is pretty powerful.
Ben Felix: Constantly nerding out, looking for little things like that. It matters. As you mentioned, it does increase the expected return that we're using as an assumption, which changes the advice on like, how much can you spend in retirement?
How much do you need to save with this stuff? Now the future is super uncertain, but I would still say that this stuff matters.
Braden Warwick: Definitely.
Ben Felix: So real quick, right now when we do our simulations, returns are modeled as random. We're defining the mean, the standard deviation, the correlations, and then we're just pumping out a thousand simulations. It's a pretty basic simulation process.
John Yang, we're going to go to our interview with him in a second. He's going to talk about what him and his student group did for their industry project and how it changed, how they came up with an improved way to do the simulations. And then once we've talked with John, Braden, you've got some really interesting analysis on how using their simulation process instead of ours changes long-term projections and what they specifically change and why.
Anyway, super interesting. John Yang is a financial engineering student at Columbia Engineering with experience in quantitative modeling markets and applied analytics. The class has now ended, but his project with Professor Michael Robbins was on a synthetic data generation project for PWL Capital, which was focused on building more realistic market scenarios for financial planning and risk analysis.
John's work on this project examines how advanced simulation methods can better capture asset behavior, tail risk, and periods of market stress than traditional models. All right, let's go to our conversation with John. John Yang, welcome to the Rational Reminder Podcast.
John Yang: Thank you. Great to be here.
Ben Felix: We're very excited to be talking to you. Real quick for listeners, I'll give a little bit of background.
Professor Michael Robbins, who's at Columbia University, reached out to me. He runs an industry project with his class every semester. The industry project basically takes a group of students and assigns them to someone in an industry or a company in industry, and they're given a problem from that person or company or whatever it may be to solve a data-based project.
So Professor Robbins reached out, asked if I thought I had anything interesting for students to work on. And I was like, actually, yeah, there's something that we have been trying to solve and we just haven't dedicated the time to do it. And so we landed on that as a project.
Braden, maybe you can real quick introduce the problem that we asked John and his team to solve.
Braden Warwick: It comes back to our expected return assumptions model. We still are using a Gaussian distribution to sample our expected returns for each asset class. And it's not a perfect approach, but it was simple and it wasn't extremely obvious to me how to improve it or what needed to be done to improve the model.
We know that returns don't follow a normal distribution in reality, but we knew that it was also going to be a big undertaking to improve that. When this opportunity came up to engage an academic partner, it seemed obvious that this was a great opportunity to improve our model. So John, why don't you tell us a bit about how you tried to tackle that problem?
John Yang: So first of all, it's great to be working with you guys for the past few months and Professor Michael Robbins actually helped us a lot in the process. And how this is going to work is I'm just going to share my screen and I will talk about the approach we took in improving the model. So like Braden have talked about before, we were essentially trying to simulate test cases for wealth planning using synthetic data generation.
And in the past, it has been done through Gaussian or normal distribution based models. What we were trying to do is to improve that and I think a good way for the audience to grasp what's going on here is we're essentially translating the forward-looking expected returns and volatilities into market scenarios that can be used as test cases. Expected returns and volatilities are very useful in looking at how your portfolio is doing or how the market is going to perform, but they're one dimensional.
There's still summary statistics. They don't actually tell us that sequence of returns that you might live through. And when planning, let's say for your retirement, that sequence matters a lot.
You're unlikely to experience, say, 6% return year over year, like in a straight line. You experience some good years, some bad years, some recovery years, some drawdowns, and during crisis assets that crash together when they're expected to diversify each other. The question for us is once we have the forward-looking assumptions, how do we turn them into realistic return paths that can be used to test the resilience of a wealth plan?
This is why we are generating synthetic data. I think one thing that I need to clarify before I start to talk about approach is that synthetic data is not randomly generated data. It's very mathematically grounded estimation that is designed to represent historical scenarios.
In our case, each synthetic data point is a part of possible return path that a portfolio could experience. The goal for us is not to say this exact path will happen, but rather we're trying to create many realistic paths so we can ask better planning questions, such as what happens if bad returns arrive early or what happens when a crisis hit.
Ben Felix: It's basically like we know returns aren't random. Empirically, you can see that by looking at real data. We had been using randomly generated returns to do retirement modeling and stress testing.
You guys were taking if we look at real historical data, what has actually happened. We know things like there tends to be a little bit of mean reversion, like you were talking about. Volatility tends to cluster.
Correlations tend to go up during bad times. You guys were taking all of those empirical realities and trying to figure out a way to generate synthetic data that reflects those properties.
John Yang: Exactly like you said, once we think about the problem that way and think about synthetic data generation, the workflow becomes, I think, more intuitive. We take the expected returns and volatilities and correlation assumptions as an input to our model. These inputs are fed into our simulation engine.
Its job is to take those assumptions and generate a portfolio return path that can be used for long-term horizon planning. The simulation engine is not trying to create new market forecasts. It's grounded in historicals and trying to generate something that's more realistic.
We are not the first ones that are doing this. Like Braden said, PWL had a baseline approach before based on Gaussian distributions. This basically utilized a multivariate normal distribution and run a Monte Carlo model on it.
And for listeners who are not familiar with what Monte Carlo simulation is, it is a common framework for generating correlated data. You can basically take expected returns, volatilities, and correlations, and the models will generate simulated returns from that normal distribution. There are a lot of advantages to use it.
Like Braden mentioned, it is very easy to implement. It is also very easy to collaborate. Also, on top of it, it's also very easy to explain.
So I think it's definitely a great baseline to start from, but it also has its issues. At the end of the day, assuming a normal distribution, a normal distribution is smooth and symmetric. We're thinking about the bell curve that look exactly the same, perfect on both sides.
And real markets don't do that at all. In reality, the left tails can be ugly, which means when crisis happens, it goes really bad. Losses can cluster.
Like when you have one bad day, it doesn't stop there. Like it goes on a strike. And those are the issues that we're trying to address with our new simulation model.
More specifically, we have three challenges that we're trying to have our model to address. First, each asset need to have its own personality. In the Gaussian baseline, you're kind of assuming everything follows a bell curve, but that's not true.
Bonds behave very different from equities that also behave very different from say oil futures. Each of them has to be like fitted independently instead of just assuming a bell curve for everything. And the second challenge is that the asset classes also need to move together realistically.
You can't just model equity without thinking about bond because think about it when equity drops, when it's like reacting to one macro event, asset classes don't move in isolation. So we're trying to model that correlated behavior in our model as well. And the third challenge that we're addressing is that the model also need to take into the bad outcomes and the tail events seriously.
For now, a normal distribution has a relatively thin tail and it's not really giving the tails attention that it deserves. That was my setup for the problem. I don't know, Ben, if you have any questions about that or anything you would like to clarify.
Ben Felix: No, I mean, I thought that was great. We're simulating each asset individually. And as you said, they each have their own personalities, their own characteristics.
And then we care about how they behave relative to each other through time. It's the correlation, but it's also stuff like everything going badly at once, which can happen sometimes even with stocks and bonds, which we've seen in pretty recent history. And then the tails, which as you said, in a normal distribution, they really don't get as much attention as they probably deserve.
John Yang: Exactly. Those are the exact steps that we're taking in our simulation process to tackle the first thing you've said about like the individual asset classes, return distribution. We first built the historical return series for portfolios that represent asset classes.
Then we model out its distributions based on its historical probability of having different returns. It's also important to mention that we are building those proxies for those asset classes using indices. For example, we're using the MSCI EAFE Investable Market Index for approximating international equities.
And we combine them into portfolios to make sure it realistically represent one asset class, for example. And once we do that, we learn the historical distributions. We model out the correlated structure between asset classes.
Through doing that, we have our shocks for the market that we are trying to generate. At that point, those data are still normalized. They're in standardized space.
And what's good about this model is it allows you to impose the expected returns that you believe to be true. And that can be adjusted from time to time when you're planning.
Ben Felix: That's really cool. So the way that you set it up, the shape of the distribution and the correlation properties and the tails and all that stuff gets defined using historical data. But then once the shape of those distributions and the way that they work with each other are defined, we can put in whatever market assumptions we want.
John Yang: Yeah, exactly. I'm also going to go through that as well and how we do that. But I think we can first look at how we are modeling out the correlation structures and the distributions of the return of every single asset class.
Essentially, we are not only thinking about modeling those two things, the correlated structure and the marginal distribution of each asset separately. When we're implementing it, we're also dealing with them separately using two separate steps. I think essentially we're asking two questions here.
First is how does each asset class behave on its own? For example, what does a typical year look like for Canadian equities? How often do we see a strong year?
Second question we're asking is how do the asset classes behave together, how they move together? For example, when U.S. equity is down, are Canadian international equities also down? Or when equity is down, does bond help offset the loss?
When we're modeling it, we also do it separately. First, we handle the dependent structure, actually. And we do that through a t-copula.
Without being too technical in getting into what copula is, an easy way to explain it is it's just a part of the model that links the portfolios, links the asset classes together. It helps you generate the correlated data by encoding it with essentially a matrix. And the reason we're using a t-copula rather than the Gaussian copula that PWL was using before is the t-copula captures the code movement at the tail better.
That was one of the challenges that we were talking about earlier. And I think what is more interesting is how we're modeling the distribution of each individual asset class. Originally, the method was just looking at asset classes as a classic bell curve.
You're assuming perfect symmetry, smooth distribution of returns. But that is not realistic. So instead of forcing our portfolio into the same bell curve, we let the historical data describe the shape more directly.
For the distribution that we're modeling, we also divide this problem into two parts. For most of the distribution, we use the actual historical pattern of the returns. We first standardize the returns.
So we're looking at the shape of the return pattern rather than the absolute return level. And then we use that standardized historical pattern as the empirical distribution. We're basically replicating the historical probability of each return level happening in the simulated data.
That works for the majority of the distribution because we have enough observation there, like in the top 90%. But in the bottom 10%, that's like when this asset class is doing the worst, that becomes a problem. This is because these events are rare and bad outcomes.
Think about 2020 when COVID happened. US equity was down, I believe, 34% in a month when it was the worst. So those are sort of outliers if you purely look at it from a data science perspective.
However, from a wealth planning perspective, it is a completely different story because those crises are actually the most important challenges to your wealth plan if you're trying to conserve wealth. For the bottom part of the distribution, we use Extreme Value Theory. More specifically, we're using a Generalized Pareto Distribution.
What it does is it is a specific distribution that is designed to fit those extreme events. So by combining those two methods together when we're estimating the marginal distribution of the return of each asset class, we're essentially using historical data where we have enough data. And then we are using like tail specific model where data is sparse.
Ben Felix: Very interesting. So we don't have enough data to properly capture the tails. And so you found a way to sort of manufacture tails that are more realistic.
John Yang: Yeah, exactly. And I wouldn't say "manufacture" because it is still based on historical. Instead of forcing it into a like specific bell shape or forcing it to look like historical exactly, which has a lot of zigzags because the frequency those events happen, we felt it with a more appropriate distribution.
And that's how we model the individual asset classes return. And once the individual return distribution and the cult movement structure are defined, we now use Monte Carlo simulation to generate scenarios. Well, Monte Carlo, what it was simply doing is creating many possible future paths instead of pretending there's only one future.
If you're only looking at summary statistics, you're essentially assuming there's only one possibility. And that is what I think is going to happen. But Monte Carlo generate many possible paths.
And on average, they match the expected volatility expected return that you specify. One analogy that I really like when I'm explaining this is that expected return, expected volatility is the destination that you set. And Monte Carlo simulation tells you many different paths that you can get there.
Ben Felix: Yeah, that's very cool.
John Yang: The next part is going to be very useful, especially for wealth planning when the market condition is changing all the time. I mentioned earlier that we're learning the shape of distribution from history, but we're not necessarily assuming the historical return is going to be the same as what it is now. Instead, we can impose a forward-looking assumption on our simulation model.
And up to this point, the model has learned the pattern of risk, the pattern of return from the history, but we do not want to simply replay history. So the model first create standardized shocks, but now you can re-standardize everything into a standard space, and then you impose the target return and target volatility on the shock sizes. Think about the simulation as a generator that's generating shocks, but it only tells you how bad is the shock relative to normal, but you haven't defined what normal is. And by imposing those mean and volatility, you're defining what normal is.
Ben Felix: Yeah, I love that. That was one of our big concerns with starting this project is that we change our expected return assumptions twice a year. We always restate them based on the way that we calculate them, and we need to be able to take this return-generating process and input what our current mean expected return or current standard deviation are. You guys solved that in a really nice way.
John Yang: It's actually a lot of fun doing that in the process. When we were trying to model it in the first place, we kind of assumed the historical mean is going to be the same as the future because we didn't think that would be an issue. But when we were doing testing, we realized it is completely off because let's say you take a set of data from the 1990s, and then you compare it to today's market scenario, the distribution of the return can even look very different.
And then not to mention the expected value and volatility looking at recent events, the volatility is through the sky, and that is not something that would have happened in the last decade. After doing all of these, the final output that we are getting is a thousand simulated paths, and each of them covering 80 years of returns. Throughout testing, we see that our results have preserved more realistic higher-order distribution features, for example, skewness and kurtosis.
For listeners who are not familiar, skewness simply means the distribution is not perfectly balanced or symmetric, and kurtosis means extreme events in the tails. Essentially, we're looking at the tails. If you look at the comparison graph on the right of the slide, the newly simulated distribution using our method are compared with the historical behavior, the historical empirical CDF, the cumulative distribution of the return, and the Gaussian baseline that PWL was using before.
You can see that the new method followed a historical shape more closely, especially in the left tail where the Gaussian model is way too smooth.
Ben Felix: Yeah, it's super interesting. You got very close to the historical distributions with your simulations.
John Yang: That's the benefit of using the empirical structure. We're not forcing it into any of the shape. It's the same as running an optimizer.
When you're trying to optimize, there are constraints that are stopping you from getting that exact shape that history has showed us.
Ben Felix: It's also really interesting how different both the simulated and historical returns are from the normal distribution. For people who are not watching, there's a red dotted line showing the normal distribution, and then there's a shaded area and a green line showing the simulated and historical returns. They're not the same shapes.
Braden Warwick: It's even interesting too how it differs from asset class to asset class. If we look at the fixed income, there's a dramatic difference between the Gaussian data and the sample data. Same thing with US equities, but Canadian equities and international equities seem to be a little bit closer.
John Yang: Not just visually, we've also ran tests on it to look at how big the improvement has been comparing to the Gaussian baseline. We believe at the end of the day, the goal for us is not to produce a more complicated model. Rather, we want something that's actually usable.
We look at three things. First, we look at if each asset class simulated return pattern look more realistic. That is what we mean by the marginal shape score.
It considers things like quantile gap, skewness, kurtosis, and downsize loss measures like CVaR. And the second metric we look at is if our model does better on the tail. This measure is mainly driven by the tail quantile gap.
And third, we also ask, does the model still preserve how assets move together? So that is the correlation structure that I talked about at the start of this presentation. We are measuring this using a basket of correlation measures like Pearson and Kendall, which some argue are more realistic. We're basically combining all of them together to capture all of it.
Braden Warwick: So I see like a dramatic improvement in the shape and the tail score, but the co-movement score is relatively similar to what we had before. Is that right?
John Yang: Yeah, that's correct. That's actually like exactly what we were expecting. Because for marginal and tail shape, that's where our method is mainly improving in. That's because we're using the empirical distribution of return rather than assuming a bell curve shape.
That's why it does a lot better in marginal and tail shape. But for co-movement, our goal is to be on par with the Gaussian method. That is because in the Gaussian method, the goal is to optimize for that correlation structure that was given to the model.
So it is doing a very good job at it. And we are actually surprised that we didn't have to compromise any of the co-movement score to get a better marginal and tail score in that sense. But yeah, it's great to have.
Well, we've only been working on this for the past five months, and this is like far from perfect. There are a lot of things that we still can do and will do in the coming semesters. For us, there are three natural extensions that we're thinking about.
The first is dynamic correlation. Right now, we're assuming a statistic correlation structure. Essentially, what we're assuming is that the average relationship between asset is stable across all environments.
But in real markets, the relationship between asset classes changes. For example, think about the recent tension in the Strait of Hormuz. The oil price, inflation, expectation, and bond yields, equities, currencies, those things are essentially responding to the same macro event, and they're interacting in the way they do right now.
But that is very different. How they interact right now is very different from more stable time when inflation fear is not a trigger for the market, for example. By using a dynamic correlation, we can model each period separately and better replicate the history with more realistic test cases.
And the second thing is something we weren't able to address with our current model, that is volatility clustering. Ben mentioned at the start of the podcast, or the start of our conversation, that some of the bad days arrive together. For example, in 2008, you don't just get one bad trading date.
Once there's a fear in a market, people sell everything and those sell-offs continue for a long time. And those clusterings are not modeled in our current approach because we're assuming independent draws when we're simulating. So that is another extension we're trying to work on.
Our current idea is potentially using a GARCH style model that deals with time series. And by incorporating the time dimension, we can solve that problem. And the third thing is not as much of an extension to our model, but rather an alternative path that we will explore in the future, that is GAN models.
So GAN is generative adversarial networks. Those are machine learning models that are the new cool kids on the block situation. People are using that to generate synthetic data.
So essentially what it's trying to do is that it is generating a set of samples using a machine learning model, and then using another model to judge it. You kind of like go back and forth and find realistic scenarios. The drawback for that is that there's still a lot of research that needs to be done.
And like a lot of machine learning models, when you're using it, you might be overfitting. You might discover patterns that are not there at all. When you're simulating it, you're essentially amplifying those patterns. That can be an issue for clients who don't want their wealth to go wrong.
Ben Felix: Very cool. You'll leave these as sort of breadcrumbs for future student groups, and this project will continue with Professor Robbins, and we may be able to implement some of these return distributions in the future.
John Yang: Exactly. I think to close my talk here, I think the main takeaway from this is that forward-looking assumptions are starting points. When you're planning for wealth to use those assumptions in planning, they had to be turned into path, and that's what we've been working on for this project.
The baseline Gaussian model that we start with is clean and transparent, but it makes market looks way too smooth than it actually behaves. And our framework keeps the same assumptions, but it produced paths that are better reflecting the messiness, I'll say, of the market. We look at asymmetries, we look at fat tails, we look at assets that are moving together during crisis.
And those are the things that matter a lot for a long-term wealth planner who are planning for their retirement or preserving their wealth. And for the next phase, we'll continue to work on the extensions that I talked about, like dynamic correlation matrix and a GARCH model, which handles volatility clustering.
Ben Felix: Awesome. I've got to read the sentence that you have here on the key model improvement, because it really does summarize it well. The t-copula plus empirical EVT framework is a stronger simulation baseline than the Gaussian model, because it better reproduces historical shape and downside risk while maintaining comparable co-movement. Very nice. Very clean.
John Yang: Thank you.
Ben Felix: Awesome. That was great, John. Thanks so much for coming on and summarizing the work that you guys have done.
John Yang: No problem. It's a pleasure to work with you.
Ben Felix: All right. That was cool. It was cool to have John's slides and kind of see what they presented to the class.
I thought that was just kind of neat to have a peek into that. I thought John was great. This was his first podcast ever, and I thought he spoke really well.
Braden Warwick: For sure. It was great. I really enjoyed working with them over the course of the past few months and really cool to see what they came up with.
The main question left in my mind after hearing his analysis is, great, this is real improvements on our modeling approach, but how is this going to impact the financial advice? How may this impact the recommendations that we're providing to clients or the financial decisions that we're helping our clients with? That's sort of the genesis of this post-analysis.
I wanted to quantify that impact on the financial advice, but I also wanted to look at their model using the DMS data. That's one thing we didn't really talk about too much with John, but they were using monthly index data with 12-month rolling time periods. The index data only went back to 2003 for all of the asset classes that we were looking at, so it was a relatively short time period.
The first thing I wanted to look at was what happens when we use the annualized DMS data that goes back to over 100 years of data. How will that change, if at all, the outcomes of his analysis? I'll say up front, ideally, I'd like to answer this question of how does this new model impact the financial advice.
I'd like to answer that directly in Conquest, but at the moment, we can't. We can't send those 4 million data points that I talked about earlier to Conquest directly via API so I can make real-time changes. Instead of doing that, I built a pretty simple lifecycle-style model where I was able to capture the basics of the lifecycle.
I modeled everything in terms of after-tax cash flows, modeled accumulation and decumulation. I modeled different ages, so different time horizons. I modeled investors that were 20 years old all the way up to 80 years old in 10-year increments.
I fixed retirement age at age 65, so that way we can model different lengths of the accumulation and decumulation, and also decumulation only. All of that was captured, and we looked at different asset mixes as well. The 60-40 portfolio, 80-20, and 100% equities.
In each case, I fixed the spending at the sustainable spending amount. I solved for the spending amount that would lead to an 80% success rate using our previous Gaussian distribution. That way, we have a benchmark that we can go back to, and it's also a realistic benchmark.
It's representative of an actual plan that we would work with our clients on and not something that is someone spending way below their means or way above their means. It's representative of a realistic client scenario. That way, we can benchmark the difference between this new model and our old Gaussian approach.
The results. First, I'll talk about the differences between the DMS data and the index data. What we found were pretty similar results, but we found the Gaussian represented the DMS data better than it did the index data.
This wasn't too surprising for me to see. It was kind of what I was expecting to see because the DMS data goes back a lot further. It's a lot more independent samples.
The 12-month rolling time periods, monthly data has data points like the March 2020 drawdown in it, which can lead to these very extreme left-tail events. That was one of the struggles that we worked with over the course of the project, was trying to account for this lack of data in the index data and the over-representation of the March 2020 drawdown, especially when it didn't really impact our long-term financial planning projections. Our clients obviously had to be able to withstand that drawdown, but beyond the behavioral side of it, it didn't really impact our financial planning projections necessarily.
Ben Felix: Well, it came back so quickly, right? That's one of the crazy things about higher frequency of return data is you can see these crazy events. Even with daily returns, it can be even crazier.
There can be these massive short-term drawdowns that bounce back. If you look at annual data, things just look a lot smoother than if you look at monthly, and then likewise with monthly versus daily. That finding, I think, does make sense.
Braden Warwick: Totally. Importantly, the new model improved on both. Even though the Gaussian was a better representation for the DMS data, this new model performs better regardless in both the DMS data and with the index data. I think it's a step forward regardless of how you look at it.
We can see the same thing too when we're looking at the tails. Maybe Matt can overlay some of these graphics on the screen for the viewers to see. Ultimately, we can see the same thing that I just talked about, where the tails of the index data were fatter than the tails of the DMS data, specifically looking at fixed income and US equities, where the DMS data was closer to the Gaussian.
But like I said, it's still a better representation of the data than the Gaussian, so it's a step forward nonetheless.
Ben Felix: For that part, basically what we found was that the new simulation method makes the return distributions look more similar to actual historical return distributions than what we had been doing previously. Is that a good summary?
Braden Warwick: Yeah. It's the same finding that John had. It's just I looked at it also through the lens of the DMS data and made sure that the results were still consistent, and they were.
Then I moved on to answering the bigger question, which is how is this going to impact the financial advice that we're giving to our clients? I started by looking at the success rates, the Monte Carlo success rate using the new model compared to the Gaussian model. Remember, for the Gaussian, I benchmarked the spending rate so that it would result in an 80% success rate.
That's what we see exactly with both the DMS and the index data. We see an 80% success rate. Then we see the new model improves that success rate by roughly 2% to 3% on average, not like an earth-shattering difference in terms of the difference in success rates.
I don't imagine it would really lead to much different financial advice if you're looking at it through that lens of the success rate only. If we were to do a financial planning update and the new success rate went from 80% to 83%, we probably wouldn't change the financial plan. I think it fits in that same lens where this change isn't going to completely overhaul the financial plans that are already in place with clients.
I personally view that as a good thing. I mean, of course, if we did find something out that drastically changed the financial advice through the lens of the success rate, we would have to implement it, but it would be a much more difficult conversation.
Ben Felix: We could see that when we do our expected returns update, which we do twice a year. It's not a huge deal. Did you look at the impact sorted by age or length of the projection?
Braden Warwick: I did, but there wasn't really anything drastic to note.
Ben Felix: Interesting.
Braden Warwick: I also looked at that success rate change over different asset mixes as well. I found pretty consistent results. Using the index data, there was a bit of a relationship.
It was still within 1%. Nothing to note. I think because the results didn't change very much at all, the average was 2% to 3%, but the bounds were a 1% change to a 5% change.
So it's really all within the normal financial planning assumptions update could change the success rates by that much. So then I went from the success rates to looking at the change in the sustainable spending amounts, and you would expect that those results to be pretty similar because the sustainable spending, basically, it's the same mechanism. The success rate is going to inform the sustainable spending amount.
That's pretty much what we saw. We saw changes of spending that ranged within 1% to 2%. Not a drastic difference.
I think one thing to note, it may be confusing because the tails of the distribution changed so much, but then we're not seeing a ton of change with this. The reason why is it's really the point on the distribution of, say, the bottom 20th percentile. If you're looking for an 80% success rate, it's the return at that exact point, the bottom 20th percentile, that's going to drive these results.
So if there's really not a big shift left or right, then it's really not going to have a dramatic impact on these metrics specifically.
Ben Felix: That's an important point. This is one of the cool things, as John mentioned, about the model that they built is that we are still using the same expected returns. We've just changed the way that we're generating the distribution around those expectations.
So as you said, we're not shifting the distribution one way or another. We're just changing the shape of the distribution to be more historically accurate, realistic.
Braden Warwick: That's right. So you wouldn't really expect a dramatic shift here anyways, because like you said, Ben, we're already capturing the standard deviation and the expected mean, and that's the same as it was previously.
Ben Felix: It's worth mentioning, you touched on this briefly, because people might be thinking, why did you guys expend all of this effort if it doesn't actually change things that much? One of the big reasons, and we haven't really gotten there yet, but we're moving in that direction. This was a first step.
One of the big reasons that we care about those higher moments of the distribution and the shape of the distribution is for other asset classes. For us to model venture capital, for example, using a mean and a standard deviation and a correlation is just not sufficient to capture the expected effects of that asset class at the portfolio level, likewise for other sort of alternative asset classes. That's something we've struggled with for a long time, because even though they're not things that we recommend, we do have high net worth clients who have allocations to those for whatever reason.
Maybe they were recommended by a previous advisor and maybe they were done as a passion project or whatever, or maybe they just wanted exposure to that asset class. Modeling that in a financial plan has been a thorn in our sides for a long time. Using this type of simulation method is going to be a huge improvement when we get to those asset classes.
I thought it was worth mentioning because we're not seeing huge changes to, small changes, but we're not seeing huge changes to the simulation results based on this better methodology. For other asset classes, it'll matter a lot more than for standard stock and bond portfolios.
Braden Warwick: 100%. I'm going to circle back to this point once I cover this next section, because it's going to become even more obvious why that is important and why that's going to be super impactful. Up to this point, in terms of the success rate and the sustainable spending amount, not drastic changes.
I feel like I make this point every time I'm on, but it always comes back to the goals of the client. If the goals of the client are purely focused on sustainable spending, then they're probably not going to receive really any different advice for the most part. They're still essentially on track to the same plan that they were using the Gaussian approach.
I think that's a bit reassuring too, to the general community that if you're just using Gaussian distributions, it's probably okay. It's not something that you necessarily need to lose sleep over. But the next section is a lot more interesting when we talk about the distribution of wealth outcomes.
For someone with goals that are aligned to their final after-tax net worth or their estate value or their multi-generational wealth, it is a meaningful difference. I think it's some pretty cool findings that I found. The first thing here that was a bit tricky to wrap your head around at first when looking at the distribution of wealth outcomes, the median net worth of the new model increased, but the mean of the new model decreased compared to the Gaussian model.
It's a bit tricky to think about, but let me try to explain this. The median, which is if we're running a thousand simulations, it means the point that there's 500 that performed better and 500 that performed worse, that increased. That makes sense because we have mean reversion, so we should expect that distribution of outcomes to narrow.
We would expect that the median would increase from that. But the mean is not calculated in the same way. And because the lower bound of wealth outcomes is zero, and at the upper bound, the Gaussian distribution or sampling from the Gaussian distribution can lead to these sort of runaway events where just imagine you have a sequence of returns that is a right tail after right tail after right tail that can really compound wealth.
So imagine a 30% return and a 40% return and a 50% return all stacked together. It's really going to compound wealth to this crazy high level that we wouldn't really necessarily expect to see in real life because there's no mechanism to pull those returns back down after a right tail return has been realized. In the Gaussian model, you see these really high wealth values that bring up the mean quite a bit.
Regardless, the financial decision should be made using the median outcome. That's how you should be thinking about the expected outcome is in terms of the median, but I'm not sure if that's universally true among financial planners or retail investors that are doing this type of analysis. So I think that's an important distinction to make is the difference between a median and a mean outcome.
The median outcome improved using our new model like we'd expect because it has this mean reversion and sequence of return modeling captured more accurately. The mean decreased.
Ben Felix: Yeah, it's interesting. That's like Japan leading up to 1990. You did have a crazy right tail event where there were just incredible, incredible returns, bam, bam, bam, but then you've got a CAPE ratio at 90 or whatever it was and returns don't last forever.
Right tail returns don't last forever in real life. But it is interesting how it changes the mean because that right tail in the Gaussian distribution with random returns is totally uncapped, but the left tail is implicitly capped because financial plans stop, like failure happens when the assets run out. You don't get a big negative number.
Braden Warwick: That's right. What I thought was really interesting about that is through our conversation with John and our work with John, we spent so much time looking at these left tail events and how to improve the left tail of the distribution. But in terms of the impact of the financial planning distributions, it's actually that upper tail that's impacted the most because it really is pulling these runaway events back down to earth.
I thought that was pretty interesting and the results are pretty meaningful too. We're seeing that the median is improving by on average about a half million dollars in final net worth and the mean is decreasing by a million dollars in final net worth. So these are definitely meaningful numbers that would impact the financial advice, especially for someone who has those types of goals like estate goals or multi-generational wealth type goals.
Ben Felix: People never believe, man. I haven't been in client meetings like this for a while now, but I know advisors still have this issue. When we show that long-term projection for someone who's 35 or 40 years old, and it says they're going to have all these tens of millions of dollars in the future, people are always a little bit skeptical.
So I mean, reining that in a little bit by improving our simulation process is great to show a more accurate number and I think it's more believable for people.
Braden Warwick: Yeah, exactly. Like you alluded to, Ben, if we're able to model this with alternative asset classes, I think that's been the biggest challenge that we've seen so far is that when we have our concentrated stock portfolios in Conquest and we're trying to answer that question, should I invest in this? Something that we would think of as like some sort of crazy investment where it's a concentrated portfolio or some sort of bet.
It's difficult to show that in Conquest because Conquest will display the distribution of planning outcomes and there's so many runaway events. Obviously we don't publish this data, but I'm sure our audience can imagine that the standard deviation on these concentrated portfolios would be substantially higher than a broadly diversified portfolio that we would recommend. So you see these extreme runaway events where there's 60, 70% returns and the upper bound just goes crazy.
So it's very difficult to present that to a client and actually make the case not to do it because it's human nature to look at that upper bound and see like, oh, I'm missing out on all of this potential growth. But meanwhile, like we know that that is statistically almost impossible for that to happen because the distribution of a concentrated portfolio or individual stock is definitely not Gaussian and the DMS portfolios are a lot closer to the Gaussian distribution than these other portfolios of single stocks or concentrated portfolios would be. And then there's also that mean reversion dynamic that wouldn't be captured either.
So the runaways become even crazier with that level of standard deviation. So we've always found it really difficult to actually show that in Conquest. So that's why I'm really excited to see what the next version of this project will look like in terms of getting those other asset classes fitted with this model so that we can make a realistic informed case for clients to look at that scenario through the appropriate lens.
Ben Felix: Yeah, that's such a good example. People have trouble thinking about how much risk should I take? Should I invest in this or that?
All that kind of stuff. One of the ways that we always come back to is let's put it in the financial planning projection and see how it affects things. People have a much easier time grasping, this will change my expected retirement date or how much I have to save or how much I can spend in retirement.
Those are much more tangible data points for people to use to make investment asset allocation decisions. But you're so right that for some stuff like, should I continue holding this individual stock or should I invest in this private markets fund or whatever? Those have historically been a lot harder to show people in our planning projection, like using that same type of approach.
Because for the reasons you've described, they often make things look good, even though we know that's not the right thing to show and the right expected result. We didn't previously have the capability to model the components or the characteristics of the return distribution for those asset classes. We're moving in that direction, which is very exciting.
Braden Warwick: Totally. And I'd really love to be able to show a visual of the expected outcome down to completely eliminate those higher bounds, because it's really not practical for planning purposes. If we're able to show clients what the expected outcome is, and obviously our audience would know that those other forms of risk are uncompensated risks.
So we wouldn't expect the expected outcome to improve in that scenario, but we would expect those downside events to increase and overall the financial plan to look worse. If we're able to just show that, then it would be painfully obvious that this is not the right move, but just including those upper tail events makes that conversation really difficult because it's human nature to be drawn to that higher potential for growth.
Ben Felix: After going through this and running the simulations, what are your main practical takeaways?
Braden Warwick: So practical takeaways, again, it comes back to goals. So if the client's goals are focused mostly on spending or almost entirely on spending, then it probably won't change the advice necessarily. And I think that's good.
It makes that transition smoother for us. We're not completely overhauling a client's financial plan. So from a change management perspective, it makes life easier.
But for people that do have any amount of legacy goals or interested in making decisions based on their final after-tax net worth at the end of their life, then this does impact the financial decision. We would expect the median outcome to improve using our new model, which is more representative of reality. And we would expect the mean outcome to decrease just because we're decreasing the bound of outcomes.
And the runaway events cause that upper bound to have a higher proportional impact on the mean. And also, I think financial planners and retail investors that are doing their own financial projections should be looking at the median outcome rather than the mean outcome. So it shouldn't change the advice.
The fact that the median improves should mean that their expectations should improve and might leave a little bit of extra net worth that they can plan for to delegate out of their estate.
Ben Felix: Cool. I think we mentioned with John, this project is ongoing. I think the analogy that I use is that John and his team have left some breadcrumbs for the next group of students that will come through.
With Professor Robbins’ guidance, he's got some pretty cool ideas for how we can make this simulation process even better and account for other interesting stuff like correlations that are time varying. When there's a market crash, things tend to crash together. Being able to model stuff like that and just improving the overall simulation process further.
And then another big piece that we did not get to with John and his team, even though we talked about it quite a bit, was figuring out proxies for alternatives. If we wanted to do this process, for example, using individual stocks or venture capital or private equity or hedge funds or whatever the case may be, a big part of that is finding good quality historical data that we can use to benchmark against. What should our simulation look like?
That's something that we're going to continue working through with the next groups of students. But it's pretty cool having this relationship, seeing the impact that it's having. We've moved a little bit in the right direction, but I think once we get through this next iteration, we will actually start deploying this into Conquest, into the way that we're running our simulated data.
At that point, it'll be having real impacts on people's financial planning decisions, which I think is pretty cool.
Braden Warwick: It's very cool.
Ben Felix: All right. I think that's it for the episode. Thanks everyone for listening.
Disclaimer:
Portfolio management and brokerage services in Canada are offered exclusively by PWL Capital, Inc. (“PWL Capital”) which is regulated by the Canadian Investment Regulatory Organization (CIRO) and is a member of the Canadian Investor Protection Fund (CIPF). Investment advisory services in the United States of America are offered exclusively by OneDigital Investment Advisors LLC (“OneDigital”). OneDigital and PWL Capital are affiliated entities, and they mostly get on really well with each other. However, each company has financial responsibility for only its own products and services.
Nothing herein constitutes an offer or solicitation to buy or sell any security. Occasionally we tell you not to buy crappy investments in the first place, but that’s not the same thing as telling you to sell them.
This communication is distributed for informational purposes only; the information contained herein has been derived from sources believed to be “truthy,” but not necessarily accurate. We really do try, but we can’t make any guarantees. Even if nothing we say is fundamentally wrong, it might not be the whole story.
Furthermore, nothing herein should be construed as investment, tax or legal advice. Even though we call the podcast “your weekly reality check on sensible investing and financial decision making,” you should not rely on us when making actual decisions, only hypothetical ones.
Different types of investments and investment strategies have varying degrees of risk and are not suitable for all investors. You should consult with a professional adviser to see how the information contained herein may apply to your individual circumstances. It might not apply at all. Honestly, you can probably ignore most of it.
All market indices discussed are unmanaged, do not incur management fees, and cannot be invested in directly. Which is a shame, because it would be awesome if you could.
All investing involves risk of loss: including loss of money, loss of sleep, loss of hair, and loss of reputation. Nothing herein should be construed as a guarantee of any specific outcome or profit.
Past performance is not indicative of or a guarantee of future results. If it were, it would be much easier to be a Leafs fan.
All statements and opinions presented herein are those of the individual hosts and/or guests, are current only as of this communication’s original publication date. No one should be surprised if they have all since recanted. Neither OneDigital nor PWL Capital has any obligation to provide revised statements and/or opinions in the event of changed circumstances.
Is there an error in the transcript? Let us know! Email us at info@rationalreminder.ca.
Be sure to add the episode number for reference.
Participate in our Community Discussion about this Episode:
https://community.rationalreminder.ca/t/market-simulations-financial-planning-411-john-yang/42316
Papers From Today’s Episode:
https://pwlcapital.com/financial-planning-assumptions-market-capitalization-weighted-portfolio/
https://pwlcapital.com/financial-planning-assumptions-factor-tilted-portfolio/
https://pwlcapital.com/financial-planning-assumptions-for-market-cap-weighted-and-factor-tilted-portfolios-methodology-guide/
Links From Today’s Episode:
Stay Safe From Scams - https://pwlcapital.com/stay-safe-online/
Rational Reminder on Apple Podcasts — https://itunes.apple.com/ca/podcast/the-rational-reminder-podcast/id1426530582.
Rational Reminder on Spotify —https://open.spotify.com/show/6RHWTH9iW7hdnA7eAg7ukO?si=fe7f60349b584026
Rational Reminder on Instagram — https://www.instagram.com/rationalreminder/
Rational Reminder on YouTube — https://www.youtube.com/channel/
Benjamin Felix — https://pwlcapital.com/our-team/
Benjamin on X — https://x.com/benjaminwfelix
Benjamin on LinkedIn — https://www.linkedin.com/in/benjaminwfelix/
Braden Warwick on LinkedIn — https://www.linkedin.com/in/braden-warwick-a40b48a3