I'm generally happy with Causal Decision Theory. I think two-boxing is clearly the right answer in Newcomb's problem, and I'm not impressed by any of the alleged counterexamples to Causal Decision Theory that have been put forward. But there's one thing I worry about. It is what exactly the theory should say: how it should be spelled out.
Suppose you face a choice between two acts A and B. Loosely speaking, to evaluate these options, we need to check whether the A-worlds are on average better than the B-worlds, where the "average" is weighted by your credence on the subjunctive supposition that you make the relevant choice. Even more loosely, we want to know how good the world would be if you were to choose A, and how good it would be if you were to choose B. So we need to know what else would be the case if you were to choose, say, A.
The answer depends in part on what is the case in the actual world. If A is a normal kind of act and if it is actually raining in Sao Paolo, then it would still be raining in Sao Paolo if you were to choose A. A simple idea is that for each complete way the actual world might be there is a unique "closest A-world": a unique world that would be the case if you were to choose A. More generally, it is natural to assume that each world determines a (more or less objective) probability measure over A-worlds, which tells us how likely A-world w would be if A were the case. For example, if A is an act of tossing a coin, then the complete truth about the actual world might tell us that if you were to choose A, then there's an equal chance of getting heads and getting tails.
But what do these A-worlds look like, more concretely?
Let's say you don't in fact choose A. Then what exactly are the most likely worlds in which you choose A?
One notorious problem arises if the world is deterministic. Any world in which you choose A then differs from the actual world either in its laws or in the distant past. Which is it? Would the laws be different if you were to choose A? Or would the distant past be different? Or would there be some chance of either? Every answer leads to trouble. I've talked about this problem in an earlier post.
Another problem, which I haven't seen discussed, concerns your motivation in the relevant A-worlds. Suppose you will choose B because you have every reason to do so. What would be the case if you were to choose A instead? Would you choose A despite having every reason to choose B, perhaps through some glitch in your brain? Or would you have different beliefs and desires that would give you reason to choose A? Again, every answers leads to trouble.
Let's start with the second horn, where we don't hold fixed your actual motivation. We then effectively assume that you have voluntary control over your beliefs and desires, which yields all sorts of wrong results.
For example, suppose you are afraid of heights and therefore (rationally) decide not to go to the cliff edge. If you had gone, then on the present approach, you presumably wouldn't have been afraid of heights. And you might well judge that worlds at which you go to the cliff edge and aren't afraid of heights are better than worlds at which you don't go and are afraid. So decision theory would wrongly say you should have gone to the cliff edge.
For another example, consider Professor Procrastinate, who is asked to write a review, knowing that if he accepts he will never complete the task. Having no control over his future motivation, he rationally declines. What would have happened if he had accepted? How could this choice have made sense in light of his beliefs and desires? Presumably the relevant worlds are worlds at which he believes he will complete the task. But why does he have this belief? If the belief itself is not a mysterious glitch, perhaps what's happening at the relevant worlds is that the Professor is really disposed to complete the review, because he somehow has a stronger desire to fulfil his duties. But then decision theory will wrongly say that he should accept.
Also, if we look only at scenarios in which the agent's choice coheres with her beliefs and desires in the sense that it maximizes expected utility, then it is guaranteed that whatever the agent chooses, she would maximize expected utility. That seems wrong. Surely it is sometimes in our power to make choices by which we wouldn't maximize expected utility.
The other horn of the dilemma is to hold fixed the agent's actual beliefs and desires. The relevant worlds where you choose A are then worlds at which you still have every reason to choose B.
This also risks making all sorts of wrong predictions.
Return to the example of the cliff edge. Suppose you have every reason to stay away from the edge, but through a glitch of your brain nonetheless move towards it. What would you do next? Presumably you would quickly turn around. You would also be puzzled and disturbed, and seek medical advice. Intuitively, these are not the kinds of situations we are interested in when we consider how good it would be if you went to the cliff edge.
We could mix the two horns: we could say that if you were to choose A, then there is some chance that your choice would cohere with your motivation and some chance that it would be a glitch. But that only combines the problems for the unmixed answers.