Episode 238: Prof. Ralph Keeney: Decision Analysis and Value-focused Thinking

Ralph L. Keeney is an expert on decision making and his passion is to help individuals and organizations improve their decision making skills. He received his doctorate from MIT, has been a professor at MIT, the University of Southern California, and Duke University, and created and managed the decision and risk analysis group in an engineering-environmental consulting firm. He is currently a consultant and lecturer on decision making.

Professor Keeney's books include Decisions with Multiple Objectives with Howard Raiffa (1993), Value-Focused Thinking: A Path to Creative Decisionmaking (1992), Smart Choices with John S. Hammond and Howard Raiffa (1999), and Give Yourself a Nudge: Helping Smart People Make Smarter Personal and Business Decisions (2020).
Keeney is a member of the U.S. National Academy of Engineering and received an honorary doctorate from the University of Waterloo in Canada.


A large portion of what we talk about on this show boils down to decision-making, and today we have our focus squarely on the fundamentals of this process. Professor Ralph Keeney joins us to discuss some of the simplest and most profound elements of decisions, and why we so often miss these aspects. Ralph is a true expert on decision analysis, and his systematic process for decision-making, as laid out in his new book, Give Yourself a Nudge, has truly life-altering potential for anyone looking to improve their future. The book and this conversation are jam-packed with insightful and understandable ideas and examples, including clarifying objectives, the vital role of our values, generating alternatives, and a comparison between decision problems and decision opportunities. Ralph lays out a great way to get started on the path to better decisions, so make sure to join us to hear it all.  


Key Points From This Episode:

  • The focus of Ralph's research and how he articulates the importance of our conscious decisions. (0:03:48)

  • The shortcomings of the trial and error approach that many of us naturally employ. (0:10:46)

  • First steps towards being a better decision-maker; using personal decisions for practice. (0:13:08)

  • Ralph explains his conception and use of the idea of 'nudges'. (0:14:30)

  • A better awareness of the often-neglected front end of decision-making. (0:17:38)

  • An explanation of Ralph's value-focused decision-making process. (0:22:25)

  • Mistakes made around retirement decisions. (0:30:30)

  • Why clarifying your personal decision values can be so difficult. (0:32:15)

  • The process of translating values into objectives. (0:36:11)

  • Ralph's important question around the different possible situations in the future. (0:41:49)

  • Simple processes and common pitfalls for generating alternatives. (0:43:19)

  • Exploring decisions that require third-party permission or commitment. (0:47:59)

  • Differentiating between decision problems and decision opportunities. (0:52:27)

  • The most important concepts for embodying value-focussed decision-making. (0:57:44)

  • The role of financial advisors in aiding their clients to make good decisions. (1:03:28)

  • Application of these ideas toward living a good life. (1:07:46)

  • Ralph's simple definition of success in his own life. (1:10:53)


Read The Transcript:

Ben Felix: This is the Rational Reminder Podcast, a weekly reality check on sensible investing and financial decision-making from two Canadians. We are hosted by me, Benjamin Felix and Cameron Passmore, portfolio managers at PWL Capital.

Cameron Passmore: Welcome to Episode 238. Another week, another spectacular guest. We're joined this week by Professor Ralph Keeney. As we were doing the research for this, Ben, I sent you this quote, and we mentioned it in the conversation we had. The only way that you can purposely influenced anything in your life is by your decisions. That just hit me. I dropped you a note. It blew us both away, that quote. That's exactly what this conversation is about.

We all make decisions. We make lots of decisions all the time. But people aren't very good at making decisions. There's a whole process and everyone should learn about this. That is the punch line. That is the point of his book that we read called Give Yourself a Nudge: Helping Smart People Make Smarter Personal and Business Decisions.

Ben Felix: Which is Ralph’s most recent book. Yeah, that quote and the theme of this whole conversation, and we were talking about this just a second ago, Cameron. A lot of this stuff is so simple, but not until you think about it. It's a whole string of those things, where like that, the only way you can purposely influence anything in your life is by your decisions. That's clearly obvious. Both of us heard it for the first time and we read it for the first time, we're blown away.

Cameron Passmore: He said so many things were so profound, but so clear and so simple.

Ben Felix: So clearly obvious. Yeah. Once you hear them. Ralph is an authority on decision analysis. He's been doing this for a long time, specifically in decision-making with multiple objectives and value-focused thinking, which is the theme of our whole conversation. One of the problems that he identifies in the book is that people are just not trained on how to make decisions. He's told us how people learn through trial and error, which again, of course, that's how people learn how to make decisions. It's rare that there's an in-depth decision-making course taught anywhere, but these skills are extremely important.

I think people will agree when they – having a systematic process for making important decisions. As we talked about, you don't want to necessarily apply this to all of your decisions, because the trade off with time there doesn't always make sense. Having that systematic process for important decisions, it'll improve your life. It just will.

Cameron Passmore: It just will. Ralph is the Professor Emeritus of Business Administration at Duke University in Decision Sciences, and also Professor Emeritus of Industrial and Systems Engineering at the University of Southern California. Mentioned the book that he wrote, which most recent book that he wrote, Give Yourself a Nudge. Phenomenal book. He's also President of the consulting firm, U.S. Marketing and Decisions Group. He has a PhD from MIT. Anything else, Ben?

Ben Felix: No, I think that's a great introduction. People will probably recognize, or may recognize Professor Keeney's name from our past discussions on goals and setting objectives for financial planning. His work led us to do our own goals survey and all that stuff, which actually, I sent that to Ralph, just to say, “Hey, we did this thing. This in your research, I thought you might find that interesting.” He did find it interesting and that actually led to this conversation. He's had an impact on us for sure and he's had an impact on our podcast listeners. Now, we get to have this excellent conversation with him.

Cameron Passmore: All right, so let's go to our conversation with Professor Ralph Keeney.

***

Ralph Keeney, welcome to the Rational Reminder Podcast.

Thank you. It’s a pleasure to be here.

Ralph, how do you describe the focus of your career’s research and your consulting work?

Well, my emphasis has always been in decision-making, but my undergraduate degree was in engineering. There were certainly no degrees in decision-making, and very few courses. Then I went to grad school at MIT and met one of the founders of decision analysis and negotiation analysis, Howard Raiffa, who was at Harvard, and I worked with him and worked with him for his whole next 50 years and mine too. The thing is, I realized the power of decision-making in the sense that it's the only purposeful way you can influence anything in your life, in your business, in your family, or in your country. You make the decisions and then the consequences occur.

It was very interesting to me. I like working on personal decisions. The style is exactly the same. It’s what's business decisions when I've worked on. The elements are the same. You can learn a lot from the personal decisions to help make the bigger business decisions, or governmental decisions, etc.

I must say, Ralph, that quote is a quote that I read in your book. When I read it, I shared it to Ben online. That quote that is the only way that you can purposely influence anything in your life is by your decisions. It hit me like a ton of bricks. I just want to thank you for that. It was a great, clarifying sentence about your book.

Well, and about decision-making, per se.

Which brings me to my next question, which is how do you articulate the importance of thoughtful decision-making?

Well, partly, it's important because that's the only way to influence your life, and we would all like to have a better life. That doesn't mean selfishly. A better life for an awful lot of people means contributions to the lives of others as well and to their community, certainly, their family, their country. That's what I think is so crucial there and why it is. Just because of that fact, which I didn't realize when I first started working out on decisions, I figured there's many ways to influence your life, but the word purposefully is key there. You can certainly influence your life greatly by acting without thinking, and particularly, if you do really stupid things once in a while.

I agree with Cameron. That quote also hit me like a ton of bricks. You read it and that's like, that’s a big, bold statement, but it's true. There's no way around it. It's the truth.

You mentioned that you've got a background in engineering. I also did my undergraduate degree in engineering, and many of our listeners are computer scientists and engineers. I think that they'll be interested in the answer to this question. Do you view good decision making, or good decisions as objective, or subjective?

Well, they certainly have subjective aspects, I think. I don't view them as objective. The sense is, why bother to make a decision? It's because you care about something. Now, many of those cares could be based on all kinds of subjective things. There's not an objective. That would be, I'm trying to figure out the right thing from externally for my decision. In fact, the right thing for me to consider in my decision, like the right objectives is what I feel is important. I think, the subjectivity is there. It doesn't mean you can't be systematic about doing it. You want to be thoughtful and organized and set up, so you can do a better job. That's what gives yourself nudges in a sense about, trying to help people with that process. I think, subjectivity would be the answer there, if I have to choose one of those two.

How do you decide whether a decision is worth optimizing?

That is a decision in and of itself. I would say, very few decisions are worth optimizing if you take the extreme. Because it would take too long, and there's just too much detail. At some point, you'd just be making slight improvements for tons and tons of more work. For a vast majority of decisions, it's not worth it. I think of what should I do with this decision as how much time might I spend on it?

For instance, if I have an important dinner engagement to meet somebody coming up in a week, they're coming to town, and let's suppose I'd never met them before, but I had heard of them and know them. Now I would say, that's worth a little bit of thought. I tried to think of how would it be good and enjoyable for me, and good and enjoyable for them. It's not just get a rest and I'm over. I could say on a decision like that, if I even gave it half an hour of thought, that doesn't mean I can't do it in a few parts. To get four hours of an evening to be a lot better than just mediocre is worthwhile. I can give a short example that indicates an answer to that.

Suppose, you do at that situation, somebody important is coming to town and you have an opportunity to meet them. You're really excited. You find out where they're staying the night before, because they're flying in. You go to the hotel and ask the concierge. You have a restaurant nearby that's really good. They’d like a good dinner for next Tuesday night. Sure, they give you one. You make a reservation. The gentleman, or gentlewoman comes to town. You meet them at a hotel, you walk a couple of blocks, you go in, sit down, have dinner. Fabulous dinner, but a disastrous evening. It was way too loud. You really didn't get to have a quality discussion. You didn't learn much about them and they don't know who you are.

Now, any of the three fundamental things could have changed that. Thinking, what is the decision? The wrong decision is, I want to eat a dinner. What you really wanted is to decide how to have a enjoyable and worthwhile evening with a gentleman and a place that serves dinner. Then what are your objectives? Well, certainly to have a great conversation. It's not just to eat. You want to have a good meal and enjoyable time, and you want them to have that. It’s one of your objectives, too.

Had you been clear on that, you might not picked a noisy restaurant, and then the alternate is, don't take one alternative ever on a decision worthy of thought. That's the decision. Is the decision to me worthy of thought? Then it's, how much thought is the equivalent of how much time in a sense?

How do people typically learn how to make decisions?

I think they don't. The reason is, other than if you want to say, learning by trial and error. That's how we develop what we do. It's natural, because we start making a few, when you're, I don't know, two or three. Depends what you want to call decisions. Certainly, by then you're making a few and some things work out well and some work out pretty bad. You hopefully develop a little knowledge in which one is do, but why they do and then you can do a better job of coming up with them.

There just isn't much formal learning on decision-making. Even you go to undergraduate schools, anywhere, like we all did . Man, there are no courses in decision-making. Even grad schools in the business schools, it might be a week in one of the courses, where they teach a couple of the fundamentals and it's a little more formal than I think it should be. I think, unfortunately, we didn't learn and that's an important skill. We could learn it. We can all improve and influence me.

Therefore, what shortcomings do people typically have when it comes to making decisions?

Well, one is they often address the wrong decision. Often just a part of it, like the example that I gave was not the best decision they should have addressed. The second thing is their first thinking is often on an alternative. They say, “Oh, I got a decision problem.” Problem is a key word there. It's a problem. “What can I do about it and correct it, or get rid of it?” They think and they come up with one alternative. They think, “Well, that might work.” Then they give it a little bit of thought. It maybe does work and it does somewhat bugs the hole in the dam. But it's not a great alternative. They should think a little bit more and come up with a better alternative.

Then a third fundamental area is they think of those alternatives before they think of their objectives. Then they're limited in their range of thinking about them. I think those are the three fundamental things and they could certainly improve. Anybody could.

It's really interesting to think about. What can people do to learn to be better decision makers?

I think there's a little bit of recognition that they would like to be a better decision maker, and hopefully believe that with a little bit of effort, they can. Since we really didn't learn any decision-making, we're at the beginning where the improvements can be rather great. I think, the best thing to do is to work on some personal decisions, because then you're not being judged by others. They don't even know what you're thinking about. They're smaller, but worthy of thought.

I like the characterization of a decision where they have thought. Maybe it's 10 minutes. Maybe it's two hours. Or maybe if it's what university am I going to and you're at that level, or what town should I live in? I mean, maybe that's worth a reasonable amount of time over a month, or longer. I think that personal effort, but particularly on the decisions like that dinner, you can use it. You can see the benefits quickly for a little bit of effort. That can really help you decide, “This isn't such a bad idea.” 10 minutes of thinking made four hours a lot better. Often, you have 10 minutes that you're just waiting for someone, so you can think.

I'm guessing that many listeners are pretty familiar with top-down nudges that are made popular by Thaler and Sunstein. How are the personal nudges that you detail in your book different?

Well, one big difference is those top-down nudges are nudges by somebody who isn't the decision maker. It’s the decision maker shoe. The difference, one thing about the nudges I talk about is you make them not just for yourself. The second big difference is they're not just affect the way you see your decision. You're nudges improve your understanding of the decision you're facing. I certainly admire tremendously what Thaler and Sunstein did. I think, they've just had a tremendous and does impact.

What they do is for instance, one of the easy ones everybody can understand, a lot of people would like to eat a little bit better in universities. They go into the cafeteria, which is, looks great and everything, and there's the pizza and the hamburgers and they get them. They have great salads, but they're put in the side in the back. What happens, you move the salads out and you put them between where you walk into the cafeteria and the hamburgers and the pizza and people will eat a lot more salads.

They didn't change the alternatives at all, but they just changed your perception of the decision. Not even your understanding. I mean, made the objective, “Well, I don't want to spend all my time figuring out what to eat. The hamburgers are close on that one.” That same thing can give them a nudge to maybe eat a little bit better if they have an interest in doing that.

Can you talk specifically about which flaws in our decision-making processes are corrected by personal nudges?

Well, I think what they do is they just really increase your understanding, each one. I mean, when you give some thought to what decision should I face? Or what decision am I facing here, you can come up with even a few alternate statements of that decision. Or think, whether you capture the whole thing. It's just systematically thinking of that issue. Then of course, when you think of what are my objectives, any additional objective you come up with that's relevant to your decision is going to help you. Sometimes an objective will come up, I've had decisions like that that I didn't think of, and then it just hits me this one and the decision’s over. It's obvious what to do now that I also recognize that particular objective.

Then the third, again, people are usually a lot better off if they have more than one alternative to consider. Because a great impact on a better decision is if you happen to think of an alternative better than the one you are going to choose, that definitely has a positive impact.

What is the neglected front-end of decision-making?

That's those three steps that I just talked about, that I mentioned earlier. I just thought that was obvious. People know their decisions. They just write down their objectives and the alternatives and then they get on with serious thinking about their decision. That's to me the front end, and that assumption that they know that, including me. I don't know mine, without a little systematic thought in it. I have a fuzzy idea. I need to make this decision. I mean, I've got let's say, a two-week vacation. Well, I need to make a decision about what to do then. That's how a lot of decisions are framed. They should be framed maybe a little bit better. You certainly want to think about, what do I hope to achieve, or get, or contribute in that two weeks? Often, we think, what should I do for vacation of two weeks?

Well, maybe I could take a visit to one of those lower 48 states. Yeah, that'll probably work. You're done. That's two weeks of your life based on not enough thought, I think, is there's really a tremendous number of things that might come up that would be useful. A little bit of thought on a better alternative would certainly enhance the quality of the decision.

The back-end, though, I should say, once you have that decision, you can either just through your best judgment, a little bit of data, or on big problems, real analysis, collecting data, building models, etc., you want to know first, how well each of the alternatives measure up in terms of each of the objectives. What are their impacts regarding them? I mean, an example, if cost is a certain thing you'd like to do minimize cost of a certain decision you're making, of course, you'd like to know the costs of various things.

Then the convenience was one. You were thinking of where you rent a chalet, or something, or where you would stay when you go skiing. Well, convenience to the slopes is maybe important one. It might be measured by distance, or some information on that. Then often, you have uncertainties about those. If you had the cost of a big research program and a company, you couldn't say, this program is going to cost 200 million dollars. It would be maybe between 150 million and 350 million with a fair amount of uncertainty there and on important decisions you'd want that there, too.

You describe your consequences. Then you've got to balance the pros and cons of those descriptions. There are formal ways to do that by developing a mathematical objective function called a utility function. Then it can be used along with the probabilities to evaluate the alternatives. Then sensitivity analysis of various things to compare the various alternatives. What I often say in the back end, what you really want to do, ideally, figure out the best alternative. If you have 10 or 15 alternatives, it's more easily often to figure out the bad alternatives. If you can get rid of, let's say, the bottom 10 of your 15 alternatives, and if you always did that on all decisions that you had 15 alternatives, even if you randomly chose among the top five, you'd be doing a lot better than people typically do with their intuitive procedures. The first alternative that comes up, if it’s “seems good enough, or okay.”

Yeah. That's the big takeaway to me is that the back-end of the decision is objectively comparing the alternatives. On the front-end of the decision, on a lot of cases people may not be making the right decision and they may not be assessing enough alternatives.

That's right. Very often, the back-end is not at all formal, particularly on personal decisions. Many of them, you don't want to do a lot of analysis. Not worth that much time. What movie are you going to go to tonight with a friend? It’s worthy a little thought. You wanted to please you and them, and maybe location, it's terrible weather, maybe you don't want to drive a long, long way. Things like that might affect it. A little bit of thought is the appraisal. That's the back end of what are the consequences here and how good or bad are they. Again, there's probably two or three, if there's tons of movie houses, that would be a reasonable one. It doesn't matter. Once you've got the best two or three. You just don't have number 20.

Can you talk about what a value-focused decision-making process is?

Sure. Values, the term I defined specifically for what you care about in making a decision. Now, there's other definitions of value in many other contexts. Given values for a decision are what you care about, that's why you're making a decision. You care about these six. That's the entire reason. Focus means since that's the reason you're making the decision, the focus is on trying to do as well as you can in achieving those various values. The steps that we've mentioned are relevant to that process. That to me is value-focused thinking and that's why it's called that. It's going to keep your eye on the ball. The ball is what you hope to get out of the decision you’re facing.

Which makes so much sense. That's one of the things I took away from reading your book is that people make decisions without knowing what their decision values are.

They often do. I mean, the first thought after they can go, “Oh, God. I got this problem. What can I do to solve it?” To me, that's a little backwards, because although you understand the problem. Let's say, you’re kids doing poor in school and 10-year-old. Obviously, you have a decision here. I want to make it such that he doesn't do poor, but there's probably other things that you'd really like to look into as why is that occurring and everything and you'd be much better at creating alternatives, knowing the real objectives you hope to encompass are included in that decision.

We've touched on this as we've gone through the conversation, but I want to ask this explicitly, just so the listeners can really grasp it. Can you talk through the steps in making a value-focused decision?

Sure. The first three are the ones that we've talked about, as you said. What is the decision I should face? What are the values I care about? Then state them as objectives after you've identified the value. Third, what are the alternatives that I want to compare? Then four out of those various alternatives, measure up in terms of each of the stated objectives. Then you've got a good description of what the consequences are of the decision. Then you've got the balancing the fifth, the pros and cons of those consequences on the different objectives for each of the alternatives. That's the evaluative part.

Then the last step is probably to do some sensitivity analysis, because there are uncertainties there. You want to know, “Well, what if I tweaked this a little bit?” If it changes the alternative a lot, that's knowledge that would help you make a better decision. Because you could have the alternative that looks like, it's really best with all these fixed assumptions might get pretty poor, if you get very far away from some of that information. Where another alternative that was pretty good with all the fixed assumptions, it might still be really good relatively if those assumptions are slightly out of line, or even more out of line. Those are the six steps I would have.

The front end has the thing we said. The back end is, what are the consequences? Then balancing the pros and cons and systematically moving the inputs around to gain a better understanding of what the consequences may be.

Super interesting. Our world is in financial planning and financial advice. It's so easy to work on the back-end to run financial planning analysis and model different scenarios and run Monte Carlo simulation and all that stuff. This is what I found so striking about your book is that those first three steps are so easy to ignore, or just get completely overlooked in jumping into the back-end analysis.

I think that's right. Although, I think, also, people do work in the firms, that's where it's much tougher. Very, very few individuals could do, or come up with the organized thinking about what the consequences are of various alternatives than you. That's exactly where you want the expertise, because you don't have it at all. It’s certainly nice if I'm working on the right problem. A better statement of investment decision than otherwise. I think, the work that you did on do people who invest basically their money in various ways, clearly understand their objectives. You did an extremely nice piece of work on that saying, no, it's difficult there, too. It's not surprising. It's a tough thing to come up with all those.

I think, probably, those people who you've helped come up with what they really care about, then you're in a much better position to give them information that's even more relevant to the decisions they have to make, with the logic for that.

Yeah, we've been trying to focus on that in the last couple of years, is not only where can we add value on the back-end analysis, but how do we nudge, to use your language, people toward making the right decisions, or evaluating the right alternatives? That's why I think reading your book was – it fits so well with the way we've been trying to think about the practice of financial planning.

Well, and your financial planning and what's nice, if they understand their objectives better. Then of course you do, because they communicated it, you're in a much better position to give them the better information for what they want. Basically, it's not two separate things, so they get the same quality of information. They get much better information not because of any shortcomings you had. Other that it's a shortcoming of information that they weren't able to provide. Now that you can help them, everything else should go a lot better on the tougher part of the decision.

How do you define the decision that you want to make?

I never found anything on that. I think it's simple in the following sense. People write down their decision and they might say, “Well, I'm not enjoying work too much, so I have a decision.” Well, I don't know what decision that means at all. There's many, many things that could mean. To try to help people be a little more systematic, I mean, I want to clarify what it means that they're not enjoying their decision. The decision is, what are the range of things you'd like to do to change that?

One of the things that helps there, rather than write down, “Well, I got a decision, because work isn't so great.” Use the word ‘decide’ is the first word in the definition, or decision, because then it’s decide and followed by what, where, when, how, or something. At least that helps structure decision. Decide what to do and where to go. You can have two of them, on my two-week vacation. That helps them. Then just practice over time that you get clear. One definition, or rather, one thing I suggest to people, if you state your decision, decide something. Show it to a few friends if you can, or even not friends, if it's not overly personal and say, “What do you think this means?” If somebody looks at the decision and it says like, decide what to do about work. They’ll say, “Well, I don't think I quite understand what you mean.” You want them to be a little bit more specific.

You gave the example earlier with the dinner out of where a decision statement can go wrong. Can you talk us through the example that you have in the book about retirement and how that decision statement can go wrong?

One thing with that one is it's often framed as should I retire or not? Now, it matters when, but they probably implicitly have that there. Maybe they're 66, so-called, they can retire. That might be the decision that they should address. Could be that it's either totally retire or not in right at that time. Many things, one can do it a little more slowly. You could put more years in there. You might want to think about that being tied to something else. You might be retiring from a particular job you have, but have another option where you could spend some time, let's say, 20 hours a week. Not 40, if that were the standard.

On something like that, which is life shaking. I mean, it just changes everything. If you've worked for 40 years and all of a sudden, you're not, that's a phenomenal change. It's certainly worthy of some thought there to get it a little clearer exactly what you should be thinking about. Then I wouldn't isolate that from, although I've heard people say this, “Well, I'll decide when I'm going to retire. If I do, then I'll think about what I'm going to do.” I think that's a tremendous mistake. Because if you thought about it after retiring and you couldn't think of anything good to do, having not retired may have been a lot better idea. It's just a real tough one there.

What challenges do people face in identifying the decision values that are important to them?

I think the biggest challenges are all those values certainly aren't present in your mind at the same time. In fact, Benjamin Franklin wrote about that, and stated that almost the way I said it, which is why I said it that way. Nietzsche, the German philosopher said, “The most stupid repeated thing in life is forgetting, or omitting objectives.” It's just true that those are so crucial. Yet, people have a hard time coming up with them. One should be systematic in the process if it's really an important decision. I should say, this is absolutely the case for me. I could not have an important decision and write down all the objectives.

I mean, one of the decisions we had a long time ago, my wife was pregnant, four and a half months pregnant. She said to me, “You don't seem really serious about thinking of her name, for the child. We should get one.” I said, “Well, we're not behind. We're right on schedule.” We knew it was going to be a boy. We got rid of half the names, i.e. the girl names, and you're four and a half months pregnant. Now, what I didn't know is he was going to be born a month later and he's 35 and totally fine now. We’re phenomenally fortunate. He was fine all the time, except those first three or four, five months.

What she said to me then, since we were behind, she said, “We should write down the objectives of a name.” She knew there was no way I could say, “That doesn't make sense.” I thought this way then. Actually, I had not thought about it. It's a great thing. Because you think about it and you have some time to think about it. I mean, you'd like a name that's understandable, pronounceable. We wanted one that was good in foreign language. I mean, you don't want to give them a name and find out in German, or something like that, it means idiots. Any of those types of things.

You'd like reasonable initials. We can all think of three letters of initials that oh, you would hate to have those initials. There's a lot of things there that are just in that case, partly then those objectives guide your thinking through what are some reasonable alternatives. That's what we did on that, thought of some reasonable alternatives and whittle them down. I mean, we wanted a unique spelling and one that people hear, sometimes if somebody we all know this, somebody says their name and you don't quite get it. It's too close to another name. It must happen to them frequently.

In thinking through the values that are important to a decision, are there any values that are universal and that people should want included in any decision that they're making?

I don't think so. Although, there are certainly a large number of decisions that have a similar one, like cost would be in almost anything that you were considering purchasing, or doing, etc., that costs money. All things equal, you'd usually prefer that to be, and decisions you're making for your family, when you say, got a couple of kids and you're growing up and everything, a large class of decisions would have separate objectives. Or you'd like child A to have a good time on this. Child B, too, your spouse and you if there were four of you. It's sometimes on decisions like that worthwhile doing. I don't think there's any universal objective for all decisions you’d face.

How do values translate into objectives?

Very easily. When I started working on this, I learned, don't ask people, what are your objectives? Because they don't know necessarily what the definition of objective is. It sounds may be technical to them. They may even be thinking of something that's relevant. They say, “Well, I don't think that's an objective.” But you don't even know they're thinking about it, so you don't hear about it. I thought, values and defining that as anything you care about was giving them the right to anything that comes to your mind regarding this decision, write it down. Because I think you get more out of them that way.

Then once it's written down, a value is something they care about. Then, regarding that, you can say, now, are there any objectives regarding this? Often it is. They say, “I care about the cost.” Well, the objective is minimize the cost. I care about how long it will take. Well, minimize the amount of time. I care about that it looks good, if it's a product and looks of the product. Then there's probably an objective there, but you maybe want to be more specific about what makes it looks good, or what characteristics, if you can. Sometimes you can't, and so you maybe just create a qualitative scale for that decision of how good it looks on a one to seven scale.

On a real important problem like, how, let's say, disrupted looking an area is in the country where a big project has happened. I mean, so much could be in there. You could even put a scale of how it might look, where there's little disruption to a big disruption of pictures. You got seven pictures from not much at all, or maybe zero disruption, down to where it really looks like an absolute mess. You could make those pictures on a real important decision, this is how it might look in various circumstances. That would help people on that policy decision, for instance, of what should we do there? How should we avoid that an impact? That would help you create alternatives, too, there.

On identifying objectives, that research that actually led me to you, which I then read your book after sending you an email, but that research had identified that people will typically omit a huge portion of the objectives that are later identified as being important to them. What are some of the ways that people can overcome that challenge?

One thing you want to have them again, feel that anything they think of that might be relevant is something they care about. That's one of the reasons to have values first. They can come up with more. But you're right, they certainly miss an awful lot of the objectives. Once you've had them do anything they can do on their own, there's a lot of questions you can ask them. Like, think of a terrible alternative. Maybe they can do that. Well, why is it terrible? Whatever they write down is going to indicate something they care about. It just has negative consequences, or could. What is the great alternative, ideal, if you could just have it no problem? Well, why is it so great?

There's a lot of things indicated in the book that stimulates thinking in that sense, about potentially good or bad consequences in the decision. Another one that you can have there is, suppose you chose this alternative, what could happen that would make this a bad alternative? Maybe they can describe that. Well, again, then why is it bad? What does that happening cause? Just a lot of ways to push people. The way I view it is the mind is a large place, even though maybe it doesn't look so large. It's just in our heads. I think it's trying to run, if I'm doing it with myself, myself all around inside my brain, looking for where there might be something that's relevant.

Then once you've done on your own thinking, you can show it and on important decisions. I certainly did this for naming a child. I asked them without showing him first, but I do my thinking first. Then what might be something you'd be concerned about then. They might give one that I'd never thought of. You can then use your friends, but you don't want to use them early, because you might anchor on them. Then that limits your thinking. Just anything that expands your thinking.

Then also, again, as Franklin said, some objectives go out of your mind and others come in. think about it at different times, if it's really important. Don't just think about it for 10 minutes. Okay, that's that. I mean, we've all had these experiences. You don’t make the decision today. You don't have to. You're making it next week. You go to bed at night, you were lying there and think, “Oh, I know another objective.” This pops up, because the mind is somewhere else at that moment. You have more time, but not that you're working all that time. It just popped up.

Near the end of your book, you have six different questions for identifying your personal strategic life objectives. We actually talked through those in last week's episode when this episode will be released. One of the other ones that I really liked in there was thinking about yourself at different times in the future and how you might evaluate the decision, or what your values are at those times. I like that one a lot.

Certainly, when you have decisions that do influence things over time, it's worthy to consider what might be the relevant considerations in those later time periods. There's different ways to divide them up, not only by age. I mean, one might be if you were young enough, but you had something that was going to go a long time. You thought you would probably get married. I mean, there could be the single period, and then married periods, and then married period with kids, if there are going to be kids and then after the kids leave. They might stimulate just different thoughts. It's just as I see it, a different look into the mind is what you're helping yourself do there.

I like visualizing the different circumstances. The thing that I really liked about it is that people like Hal Hershfield have done a lot of empirical research on how visualizing your future self affects the way that you think about decisions, like financial decisions. From that perspective, I think it's a really insightful way to generate values for yourself. Can you talk about some of the common pitfalls that you see in generating alternatives?

Sure. There are a lot. I mean, the biggest one is not spending enough time. You want to spend a little time thinking about them. Another very big one is thinking about them before you think of your objectives. Because the only reason I care about any alternative is how well is it going to meet my objectives? If I don't have my objectives in mind, I'm not going to be very good at creating alternatives. I might be lucky and I could come up with a very great one. That's a crucial thing.

Then knowing that the only thing that makes alternatives good or better is how well we achieve the various objectives. I think, the foundation for stimulating thinking about alternatives is the objectives. You might even prioritize your objectives. If you have seven objectives, three might be, what you’d say is the more important ones. The other ones might be really matter, depending and how they are. Then you just take it. Now, what would be an alternative that does great in terms of this objective one? Just one objective. It's easier to think of that, because it's not the whole problem. It's simple. They can come up with one. Well, what's another one? You do that a little. Then you take the second objective. This looks like it's a pretty important one. Then a third, and you stimulate thoughts on it. Well, that in a sense, might partly frame the range of some of the alternatives that you would consider.

Then you can ask them questions like, well, how can you maybe combine what you came up with one of the alternatives when thinking of the first objective, and one with the second? You often can put them together, or tweak one of those alternatives. You just come up with more that way. The answer to the question in simple terms is you want to use the objectives to stimulate thinking about alternatives, because the only reason you care about alternatives is how will they achieve your objectives?

That's so obvious when you say it, but not obvious before you've thought about it.

Well, I think it's true. It wasn't my intuition all my life, either. It is very, very simple, a lot of those things in decision-making. But we just, again, started making decisions at two, or three and did whatever habits took us and we didn't systematically give much thought, me included, to all the various small details that we're talking about here that can have a very large effect.

As pitfalls for generating alternatives, you mentioned not spending enough time and not knowing what your objectives are, is there a systematic process that you recommend to successfully generate relevant alternatives?

Well, it's somewhat systematic to take each of the objectives that you have, because you do that first. Then go through just on objective one and think, what are some alternatives that would do great on this objective, for this decision? You can do that for each of them. Now you have, I would say, a wide range characterizing the large box of where alternatives would be, because they should be something that contribute to each one. That's part of the process. Then do your own thinking first, but then certainly, in a lot of decisions, I'd like to talk to friends, and then other class of decisions to people who have some expertise, like on the financial decisions, once I had a clear statement and understanding of what my objectives were.

I don't know 17 alternatives that might do great on some of those, but you would. Your comments on some of those that would be great, might be the best alternatives by far. I wouldn't judge myself as being great at creating those, but I could help stimulate the creation of them. Both I and you is the investor helping me, or the investment firm helping me, excuse me, could each do better.

What do you do if an alternative requires authorization, or commitment from another third party?

This is a thing I learned very early, because I did realize when I was a teenager and even before that. I mean, anything I wanted, the only way to get it is make a decision. In those cases, a lot of those decisions somebody has to control, especially when you're a kid. All you have to do is create the alternative, so that it's good for the authorized decision maker, also better for them. Now, I can give tons of professional cases, but I'll give one and then I'll give one other one that's, I think, very relevant.

I mean, my first job, I went to MIT for grad school and got my PhD there. Then I interviewed and I got a job as a electrical engineer and operations research systems analysis. The civil engineering offered me a job. Usually, new faculty teach the worst courses, or they’re courses that no one wants. They're not necessarily the worst courses. I didn't particularly want to do that. And so, I made an appointment with the chair of the department. Went and talked to him and said, “What are your objectives for this department?” He said, “We're not a big state school. We can't compete with some of the civil engineering schools on basic civil engineering. We need to be more in public systems analysis. Yet we have just more straight civil engineers here.” Very congenial talk, and so my contribution was that.

I went away. In a week, I came back and said, “I thought about your objectives. I could develop a course on public systems analysis.” I was really new then. Tons of real examples, and put it together and teach it in the first term. They said, and it seems to me that that would probably be enough teaching component. Supposed to do some research, too, for a first term. He said, “Yeah, I think that is plenty. That sounds like a great idea.” I met some objectives that he had and that I had. I never got those courses I didn't want to teach. That was creating alternative that was better for him, too.

Now, the other one's a smaller one, but has had a tremendous impact, way more than I thought. Our son came home when he was 10-years-old one day. Opened the door, came in and said, “Dad, there's a special TV program from 8 to 9 tonight. Can I watch it?” I thought a second. I said, “Who controls whether or not you can watch it?” He said, “Well, you do.” I said, “Yes, I do.” I said, “Do you have any homework to do?” He said, “Yes.” I said, “How long would it take you to do it and do it as well as you can?” He thought. He said, “About an hour.” I said, “What if you didn't ask me that question, whether you could watch the television show tonight? Rather, you said, dad, I have about an hour's worth of homework to do. If I go do it as well as I can do it and finish it, can I watch special TV program from 8 to 9 tonight?”

He looked up and said, “You'd say, I can watch it.” He said, “You're absolutely right.” Who would control whether or not you could watch that TV program? He thought quickly, and a smile started. Very big. He looked up and said, “I'm going to go to my room and do my homework as well as I can do it.” He got it. I mean, all parents want is the kid to stay out of trouble and not do really stupid things and do well in school and some stuff. He had tons of power. The trouble is most kids feel they have no power.

He uses that on me still. It's not using on me, because everything like that is better for me, too. As adults, I mean, you do that. You want to do something with another adult, you think, what can we do that’s good for them and good for me? It's just a natural thing. That's the answer to the question of how you can get the authorization by the person who's the decision maker.

Yeah, those are great examples. One of the other things that I found to be quite profound in your book is this distinction that I'm going to ask about. Can you talk about the difference between a decision problem and a decision opportunity?

That distinction actually comes from a book I wrote in, I guess, 1991, called Value-Focused Thinking. Decision problems are usually problems. We sit around and we don't have any decisions, and then something happens, it’s not so great. I mean, we get sick, the toilet stops working, you got to get a plumber. I mean, all kinds of things that there's a problem. It is a problem. The only way to fix it is you make a decision. I think the term decision problem comes up and that's reactive thinking which we have to react. I mean, if the toilet doesn't work, you need to do something about it. You need to react.

Given that, your life's going along, and then you get a problem, like lose your job. The quality of your life drops briefly. You try to get another job and hopefully, you can get it back to where it was, or maybe even higher. That's decision problems. Decision opportunities and problems occur for circumstances beyond your control, usually. Decision opportunities are, your life is going along and you think of something that would make it better. Then, so you think the only way to make it better is make a decision. That's the decision opportunity, because you created it. It didn't fall upon you for negative reasons, and your quality of life doesn't drop just because you thought of that.

Now, if you can think of alternatives, it's something you wanted, that alternative will improve the quality of your life. That's the difference between them. One, you create and it's proactive; those are decision opportunities. Decision problems occur because of circumstances and it's reactive. Then the other distinction is you can't in general, improve your life by solving decision problems. I mean, you get it back to where it was, so it's an improvement from after the problem. It's not an improvement to… before the problem, whereas decision opportunities are improvements relative to the status quo of today.

I love that. Thinking through it, it just highlights the importance of knowing what your values are, or what your objectives are.

Yes. I think that's it. I mean, sometimes one of those thoughts occur. You don't have to know your objectives to make a decision opportunity. You can just think of, wow, that'd be great to do. Somebody might even tell you something they've done, or say something, and you just realize, the idea comes in. You might not have stated all the objectives. I mean, it could be a great trip to Churchill, or anywhere. That it just comes up, you'd like to see the polar bears and everything like that. Then you think, too, “I'd to do that.” Well then, you need to be a little more specific on what doing that means. It might not be just go to Churchill. I mean, you could change a lot when you start thinking about other objectives that could be tied into that same decision that you might make.

How common is it for decision makers to actually create decision opportunities as opposed to just solving problems?

I don't think it's that common. A lot of people just are not active in managing their life. They're reactive and the problems occur, and they just try to solve them and keep things going. I think, there's possibly a little bit of a reason, because it's always useful to have a specific term for a concept, because it's more likely to get understood and acknowledged and recognized. I don't even think there was a term, or I didn't know of it, for what I call a decision opportunity. I'm sure others have done these things throughout their lives. They have a specific term that differentiates it from decision problems. Somebody probably also did that, but I'm not aware of it.

In general, most people are not even aware of that distinction. Since decision problems, a lot of people, I think, think of decisions. “God, I don't like them.” It always occurs, because it's something I wished didn't occur. They think, problems, who wants problems? I don't want decision problems, but I do want decision opportunities.

We talked about the distinction between those two, but can a decision problem be converted into a decision opportunity?

Sometime it can. You could say the example, where I went to the chair of the Civil Engineering Department was in the spear of that, in the sense that I took a job, so it was clearly evident, I would be assigned something to teach. It's usually, you're brand-new. It's the things no one else wants to teach. You could say, that was a problem was handed to me. Teach something I don't want to teach. You can often think about that and get out of those things, I believe.

To wrap this all up, what concepts that people need to be able to apply to become value-focused decision makers?

Basically, the things we've talked about here. In the narrow, it's for any decision you have, think first about, what do you care about? Those are what I define as the values. Then state them as objectives. More broadly, you want to go through the front-end systematically that I talked about, and then whatever is appropriate for the back-end. Again, for many decisions, it's no systematic analysis, or anything like that, and it wouldn't happen, and very few people could do it. You need somebody to help you, just like very few people can do the back-end of comparing a lot of alternatives, financial alternatives for investment.

In terms of three objectives over the next 10 years, if you only had three. They just can't do that. They can't build a model and run it through and well, what happens if there's a certain problem in the world? Well, they don't know what's going to happen to these investments. That's not surprising. I don't either. I don't have the expertise there. I think that's the key three things that you really want them to do is the front-end well, and that's good value focus thinking.

As well as then great decision opportunities, which you create with your values, i.e. something you want comes to your mind. Then also, the other thing is, how do you get the authorized decision maker? That's so crucial in all kinds of things at work. We all have done that. I mean, most people, if you have a family, certain family thinks you need the authorization and you want to do a family thing. Everybody needs to sign on. Now, if two kids are 10, you don't need to – you don't sign them on, typically. You tell them what they're doing, in many cases. Certainly, as things go on and spouses, I mean, a lot of those – both people should sign up if you’re going to do something together.

We've talked about a lot of information then. It's this systematic process to apply to making better decisions, which is profound, but it's also a skill set. How do you suggest people go about learning how to apply what we've talked about to their actual decisions?

That's a great question. The fact that it is a skill, I think it's the same. Often, when I talk about it, I say, it's like tennis. That's a skill. Could be like playing the piano as a skill, anything. But tennis, most people understand the basics. I said, you could read everything written by the experts about what you need to do in tennis. I mean, the first serve, the second serve, the forehand, the backhand, the lob, and know how to do it. You could even talk to some of the – if they were available, the great tennis players, and ask them for hints to play better tennis.

If you only went out and played 20 tennis games a year, you would stink. You can't do that. You wouldn't do anything, hardly. That is because you need to practice. You practice in the components, like the serve. Then later on, break it into first and second serve, maybe. You practice all these things. Then when you play the game, you will be a lot better. I think, it's the exact same thing with decision making, as it's a skill, you need to practice.

I didn't quite get this years ago and more, learning this is we used to think, people can write down their objectives for complex problem. As you showed and stuff, you just can't. One of the things, the place to practice is personal decisions. They're not as complex. They don't require as much time. No one's judging you and would judge you because you missed an objective there that that they happen to think of. That's where you really get to practice. If you do this a few times, and the nice thing about the nudging, you practice and you think of one objective that you missed, it might lead to an alternative is better than any you thought of.

You can see the benefits of not too much time of the practice, exactly on some things that mattered. Then you do that. You do become naturally more skillful, because practice improves your skills. That's the way I view it. It's the personal decisions. In fact, it happens that in the month and a half, I'm going to teach my fourth course, that is really all practice. It's a four-hour thing at a professional society meeting. It’s going to show them the first hour, ask them to do some things. Write down some decisions you're facing, layout your objectives and the alternatives you consider. Then go through the things equivalent to what you did on the objectives of financial investment, and show them that when they think about it more deeply and systematically, they miss an awful lot in their initial structure.

Just by going through being a little systematic and using straightforward ideas that aren't tough, they're going to do a lot better job. Hopefully, they can see that and the message is practice is going to help you. Knowing how to do it isn't. It'll help you know what to practice, but it alone is not going to help you make better decisions.

Sounds like a great course. I'd like to go back to your comments around outside perspective. Ben and I are financial advisors. What role can an outside perspective, like a finance advisor play in making good decisions?

Making good decisions for whom? Themselves, or their clients?

For the clients.

Well, in general, I would say, the better you understand their decision, the better you can help them. That seems like it's obvious. Helping them understand the decision they face, because they know what – somewhere in their mind, what they care about is in there. Your understanding of all that is going to put you on a, I think, a much better page situation to help them. I think, the decision that the clients would face is pretty similar. What's the decision? Well, it's investing their funds, where to invest them. That's pretty well-known for that particular decision.

What's missing is their objectives and their relative importance on some of those and I'll come one more to that. The alternatives are the things you control, or know about. They have a general idea, but the specifics, they don't. If you can help them and I think the work you even did helped them a great amount is going to be very useful to you to have a better impact on what they hope to achieve.

Now, the one separate thing I was going to say, one of the biggest mistakes people make with objectives is to say, they list them. Well, I want it to be high quality and cheap cost, just to take a hypothetical situation. They'll ask them, well, what's more important? The quality of the cost. People will answer that question. Let's suppose they say, well, the cost. Suppose, it's buying a TV. I'll say, okay, well, here's one TV. It costs $900, but doesn't work very well. Here's another one, $920 and it's fantastic. It does all kinds of things. Which do you want? “Oh, I'll take the 920.” Well, you just told me, cost is more important and you chose the most expensive one. The mistake is so much made, the importance of an objective doesn't depend on its name. It depends on the range of consequences over that name that are relevant to the decision.

In this trivial example, the range of consequence was $20 and the range of quality, I just made it up, but it was very large. $20 is way less significant than that large difference. I could give a time and cost on an airplane would be the same thing. Well, I want the cheapest, and the cheapest takes you three days and for 50 more dollars, you could go in six hours. Probably wouldn't be worth it for most people. That's a tremendous error people make in their thinking. It would be relevant to sometimes explaining some things to clients that you deal with, who can get mixed up in that, because they can get hung up. Well, the most important thing is, is, is. It may be is given the big range, that's theoretically possible. But if the alternatives have a small range on that particular, that isn't the most important thing.

Interesting. Is that like managing tradeoffs across objectives? Is that a way to describe it?

Yeah. That’s the tradeoffs and it's the tradeoffs of how much of each of the two, that's what a tradeoff is. That is handled explicitly with the tradeoffs. Like, is $20 more important? You might say, how much is it to increase that quality? They said, “Is it worth a $1,000?” Oh, no. Not a 1,000. “How about 500?” No. “How about a 100?” Oh, yeah. It's worth a 100. It's somewhere in that range and maybe it's worth 200. Well, in the example I gave, it's only $20 difference. The quality swamps the $20 there.

Right. At the risk of being repetitive, because this has been a theme throughout our conversation, how do you take everything that we've talked about and apply it to living a good life?

Well, I think it's useful to think about what one's life's objectives are. I mean, what you want to get and what you want to contribute. One of the things, I did this years ago, and they're structured in my books, but I thought of three things a lot of people say, the most important thing in their life is their health. If you had fabulous health, but nothing else in your life, nothing happens. It's just not there. The three fundamental things that seem to me you need to even have a life is you need in current time, you need health, and better health as good. You need time, so you can do things. You need, typically money in modern life, so you can do other things.

Even if you did extremely well on those three, you don't yet have anything happening in your life. It's what you can do, given you have those things. You follow through on the objectives. I mean, mine are what I get out of life, enjoyment and fun, etc., that part. Then the other part is the learning, the understanding, the contributing. Then what do I contribute to others in life? Then, you know, I like to maybe think that I've helped some people make better decisions. I have had sometimes people come up and tell me, they read something I wrote, and it had a major impact on a crucial decision in their life. They describe it to you and you couldn't feel better. Just can't. Then you want to contribute to your family and friends, and then maybe a little more to the world at large.

Those are my really fundamental objective. Now, their means from those health time and finances that get to there. I think, it's reasonable to give some thought to life objectives, relaxing when you're doing it, and you'll change them and modify them a little bit over time, but then they get pretty stable. Because if they're not stable, they're not life objectives.

I followed your process in the book of how to form your strategic life objectives. I did my own, which was a fun process to go through. That was great. I can definitely tell you that your – well, you saw the big survey project that we did, where we collect all those financial goals. Your papers on that, even before I had discovered your other work, your papers on that were impactful to us, but also to the 300-plus people that did the survey and the many more thousands that heard us talk about it. Yeah, you definitely had an impact on a lot of people through that.

I’m fortunate on that. It's great that you did that. It's such a great thing to have done. I'm sure a lot of people in the financial field were surprised, and some are probably no sayers. No, no, no. If people know what they want.

Our final question, Ralph, and I'm guessing, since both of us have read your book, we're guessing you have a pretty good answer to this. How do you define success in your life?

The technical term is, how well I'm measuring up, in terms of those things I said are the fundamental objectives of my life. I mean, what I get and ask emotionally and intellectually and psychologically, and what I offer to others and the contribution to society. That's how I measure them. Now I don't quantify this, nor do I, day-to-day have a scale, how did I do on that, but it's the thing that drives you. You want to make it interesting and have fun and contribute to others. It's not real high sophistication. Change things, it's maybe just be a little more coherent of exactly what are the fundamental things you really care about? Those are the ones for me. I think, they'd be similar for others. Now, there's specific ways they can contribute to others lives and the specific things they might want in theirs could be different for mine. The general, I think, would be pretty similar.

Yeah. I think that's true. I think it lines up with a lot of the stuff that's come out of positive psychology in the last little while. Your objectives and my objectives match up pretty closely to what generalizes and what seems to be important to people leading good lives.

All right. Well, Ralph, that's the end of the questions that we had for you. We really appreciate you coming on the podcast. This has been an excellent conversation.

A pleasure for me. One of my objectives is to learn things, and I certainly did. Another is to have a nice time, which I certainly did. Thank you very much. And did because of you two.

Is there an error in the transcript? Let us know! Email us at info@rationalreminder.ca.

Be sure to add the episode number for reference.


Participate in our 23 in 23 Reading Challenge:

23 in 23 Reading Challenge — https://rationalreminder.ca/23in23

23 in 23 Reading Challenge on Beanstalk — https://pwlcapital.beanstack.org/

Participate in our Community Discussion about this Episode:

https://community.rationalreminder.ca/t/episode-238-prof-ralph-keeney-decision-analysis-and-value-focused-thinking-discussion-thread/21795

Books From Today’s Episode:

Give Yourself a Nudgehttps://amzn.to/3WKMwZx

Smart Choiceshttps://amzn.to/3Ho0vP6

Value-Focused Thinkinghttps://amzn.to/3HgC7yH

Decisions with Multiple Objectiveshttps://amzn.to/3DywYRH

Links From Today’s Episode:

Rational Reminder on iTunes — https://itunes.apple.com/ca/podcast/the-rational-reminder-podcast/id1426530582.
Rational Reminder Website — https://rationalreminder.ca/ 

Shop Merch — https://shop.rationalreminder.ca/

Join the Community — https://community.rationalreminder.ca/

Follow us on Twitter — https://twitter.com/RationalRemind

Follow us on Instagram — @rationalreminder

Benjamin on Twitter — https://twitter.com/benjaminwfelix

Cameron on Twitter — https://twitter.com/CameronPassmore

Prof. Ralph Keeney — https://ralphkeeney.com/