Decision theory comes in many flavours. One of the most important but least discussed divisions concerns the individuation of outcomes. There are basically two camps. One side -- dominant in economics, psychology, and social science -- holds that in a well-defined decision problem, the outcomes are exhausted by a restricted list of features: in the most extreme version, by the amount of money the agent receives as the result of the relevant choice. In less extreme versions, we may also consider the agent's social status or her overall well-being. But we are not allowed to consider non-local features of an outcome such as the act that brought it about, the state under which it was chosen, or the alternative acts available at the time. This doctrine doesn't have a name. Let's call it localism (or utility localism).
The other side -- popular mainly in theoretical philosophy -- puts no such restrictions on the individuation of outcomes. In the simplest version of this view (Jeffrey's), an outcome is a conjunction of an act and a state. In any case, on this view the specification of outcomes must not omit anything the agent cares about, and we don't assume agents only care about a fixed set of local features such as monetary payoff.
When people criticize decision theory, the target is almost always localist decision theory.
For example, a key point in Kahnemann and Tversky's influential criticism of decision theory is the observation that people care not just about what they will own in absolute terms as the result of a transaction (e.g., how much money and how many concert tickets), but also about whether they gained or lost certain goods through the transaction. Localist decision theory is blind to these aspects of outcomes. Non-localist decision theory is not, and can easily accommodate these supposedly problematic preferences. For example, we can simply say that selling a concert ticket for $50 has significantly less utility for an agent than not buying the ticket for $50 in the first place.
Other well-known examples are putative cases of intransitive preferences (as discussed e.g. in Broome 1991: 100f.) or cases where agents care about fairness or risk: Weirich 1986 nicely explains how e.g. Allais's and Ellsberg's counterexamples to decision theory are really only counterexamples to localist decision theory.
Personally, I find some of the counterexamples to localist decision theory completely convincing. Actual people don't just care about local features of outcomes, and that does not make them irrational.
The puzzle is why anyone ever thought localism is a good idea, and why so many theorists still hold on to it.
As I mentioned in the beginning, there is sadly little discussion about this. Anti-localists like Weirich or Broome or Joyce have rightly pointed out that localism is unmotivated and problematic, but there has been little response from localists.
A welcome exception is chapter 4 of Lara Buchak's Risk and Rationality, which is basically a defence of localism. Buchak's main objection to non-localist decision theory is that it underconstrains rational choice. More specifically, the supposed problem is that for any choice behaviour whatsoever one can construe non-local utilities and probabilities that sanction the behaviour as conforming to (non-localist) decision theory. In fact, one can construe many radically different utilities and probabilities that would do the job. Consequently, Buchak argues, non-localist decision theory loses both its normative force and its status as a template for defining beliefs and desires via an agent's choice disposition.
Let's begin with the problem of normative force. Here it matters where we want to see that force applied. An obvious location is at the interface between beliefs, desires, and choices: decision theory says that if you have such-and-such beliefs, such-and-such desires, and such-and-such options, you should choose option so-and-so. Here we assume that the beliefs and desires are simply given -- perhaps treated as theoretical primitives as e.g. Eriksson and Hajek 2007 have argued for the case of belief. This normative application of decision theory clearly doesn't depend on localism.
Alternatively, we might want to say that decision theory puts normative constraints directly on one's preferences, without linking them to beliefs and desires. Again, non-localist decision theory does endorse such constraints -- for example in the form of the Bolker-Jeffrey axioms.
The only loss in normative force comes when we seek direct constraints on choice behaviour alone. Localist decision theory prohibits behaviour that manifests risk aversion or loss aversion, non-localist decision theory does not. Without bringing in beliefs, desires, or preferences, non-localist decision theory plausibly allows for any choice behaviour whatsoever. But is that a problem?
If we follow Hume and hold that rationality puts no constraints on one's basic desires, I'd say it clearly isn't. If someone has a basic desire to choose option A in decision problem 1 and option B in decision problem 2, then surely the right thing to do by the lights of their beliefs and desires is to choose A in problem 1 and B in problem 2.
But we don't need to go all the way with Hume. I have some sympathy for the idea that rationality puts constraints on one's basic desires. But not the constraints imposed by localism. Localism in effect assumes that all basic desires -- at least to the extent that they are represented by the agent's utility function -- pertain to local features of outcomes. It rules out rational agents with a basic desire for risk, for fairness, for integrity, for doing what one would like everyone to do, for living a life that fits a certain pattern. We may want a special label for such desires, but "irrational" is really not an useful label. (I'll use "non-local" instead.)
The point is that giving up localism does not mean giving up all constraints on the features of outcomes that may be tracked by an agent's utility function. It merely means allowing utilities to track some non-local features.
In fact, it seems to me that non-localist decision theory provides a much better framework to think about the rationality of desires and choices. For example, consider the familiar observation that it's good to be risk-neutral if you're playing poker or trading stocks: here a propensity either for or against risky choices is likely to deliver worse results. In itself, I don't think this makes it irrational to be, say, risk-seeking at a particular point in a particular game of poker. After all, you may well play for the thrill and not care about money. There's nothing intrinsically irrational about that. However, many people who actually play poker or trade stocks care a lot about long-run monetary payoff. They want to be the kind of person who earns a lot of money in the long run. That's a legitimate desire. For those agents, risk-seekingness in individual choices is irrational insofar as it clashes with the "higher level" desire about the person they want to be. The irrationality lies in the tension between different desires. Maybe that's not the most illuminating story, but it certainly strikes me as a lot better than the localist story which ignores both the desire for long-run performance and the desire for risk in a particular choice, since neither is local.
In sum, on two of the three normative applications of decision theory, localist and non-localist theories are on a par. On the third application, non-localist theories are preferable since the normative claim of localist decision theory is not only incompatible with the Humean doctrine of desire, but should also be rejected by those who want to put substantive constraints on rational desire.
Let's turn to Buchak's second worry: that non-localist decision theory is unsuitable as an "interpretative" theory for defining beliefs and desires from an agent's choice disposition. The reason, which I'm happy to grant, is that without prior constraints on utility one can generally find many radically different beliefs and desires that would fit the agent's choices.
In response, the first thing to be said is that I don't know anybody who seriously believes that one can define beliefs and desires in terms of choice dispositions alone. Even philosophers who (unlike e.g. Eriksson and Hajek) like functionalist theories of mind generally accept that the relevant functional role of beliefs and desires is not exhausted by the production of behaviour. In particular, there's also an important link between attitudes and perceptual input. (See e.g. Lewis 1974 or Lewis 1983.) So we really shouldn't expect beliefs and desires to be determined by choice dispositions, and it's not a problem for non-localist decision theory if that becomes impossible.
Secondly, let's grant for the sake of the argument what nobody believes: that beliefs and desires can be defined by choice dispositions alone. It's true that this won't work if we don't impose prior constraints on beliefs and desires. But it is still a long way from this observation to an argument for localism. Why not impose other constraints on basic desires? Intuitively, many desires that are allowed by localism are quite bizarre -- for example, a basic desire to either own 48526 concert tickets or an amount of dollars that is divisible by 217. Why not rule out desires like this and allow for some intuitively sensible non-local desires such as a desire to maintain one's integrity or a desire for risk?
So like Buchak's first worry, the second worry also hardly provides a reason for siding with localism.
All that said, I can see a certain appeal in localism -- especially in the extreme versions on which utility is a simple function of money or happiness. This turns decision theory ("Expected Utility Theory") into an elegantly simple model with great predictive power. The model may even have a rough fit with real people's behaviour under certain conditions. If we want a tractable model of those situations, the simple model will be much better than a more adequate model which allows for the whole range of what real people really value. But of course the simple model will often reach its descriptive limits, and it has zero normative force. For other contexts, slightly more complex models may be better that take into account a few other things people may desire. For example, I can envisage Buchak's "Risk-Weighted Expected Utility Theory" to be useful for modelling certain contexts where attitudes towards risk play an important role but we still want to bracket other non-local desires. (I find the reformulation of Buchak's theory in Pettigrew 2015 more perspicuous though, because it correctly identifies a desire for risk as a desire, not as a third kind of attitude besides belief and desire.) But again this model will have descriptive limits, and has little normative force.
For no context, however, do I think it makes sense to draw the line around local features of outcomes.