The absoluteness of consistency
A somewhat appealing (albeit, to me, also somewhat obscure) view of mathematics is the pluralist doctrine that every consistent mathematical theory is true, insofar as it accurately describes some mathematical structure. I want to comment on a potential worry for this view, mentioned in (Clarke-Doane 2020): that it has implausible consequences for logic.
Let's assume that first-order Peano Arithmetic (PA) is consistent. Let Con be the arithmetized statement that PA is consistent. By Gödel's Second Incompleteness Theorem, PA + ¬Con is consistent. But this theory is false, as we've assumed that PA is consistent. So not every consistent theory is true.
More generally, the worry is that pluralism seems to imply that there is no fact of the matter about which theories are consistent, given that consistency statements are mathematical statements – just like, say, the Continuum Hypothesis. Doesn't pluralism imply that there's a structure in which PA is consistent and one in which PA is inconsistent, and that nothing favours one of these over the other?
Let's begin with the formal, arithmetized version of the worry.
We know what kind of structure is described by PA ∧ ¬Con. This theory describes non-standard models of arithmetic with extra "numbers" besides the standard natural numbers. The arithmetical statement ¬Con is short for ¬∃xPr(x,'⊥'), where Pr(x,y) is a formula that "expresses the proof relation of PA" in the sense that it is satisfied by two standard numbers iff the first number codes a proof in PA of the sentence coded by the second number. ¬Con is true in the structures described by PA ∧ ¬Con because the formula Pr here holds between some non-standard "number" n and the code of '⊥'. But this non-standard "number" n doesn't really code any proof. So if we think of PA ∧ ¬Con as talking about the structure in which it is true, we shouldn't read 'Con' as saying that PA is consistent.
I want to say essentially the same about the non-arithmetized version of the worry. Yes, the words 'PA is inconsistent' or 'there is a proof in PA of ⊥' can be interpreted as saying something true. That's trivial! But words don't become true just because they can be interpreted as saying something true.
The point is that 'consistent' has a fixed meaning in (technical) English. Given this meaning, 'PA is inconsistent' is false. It doesn't become true by being included in some consistent theory.
The same is true for most mathematical expressions. If someone, due to a calculation mistake, claims that 13 times 154 is 2004, they haven't said something true – even though '13 x 154 = 2004' is part of some consistent theory.
'Set' also has a fixed meaning, but its meaning might be less determinate. Some structures are definitely not candidates for the structure of sets, but arguably there isn't a unique structure for which the word is reserved. We can study ZFC + CH and ZFC + ¬CH, and the conventions of (technical) English don't dictate that only one of these describes things that are properly called 'sets'.
We should separate the somewhat appealing metaphysical doctrine of pluralism from the unappealing semantic doctrine that mathematical expressions have no fixed meaning.