Appraising evidence OR how to get closer to the truth

[feralimal]

In a recent post: The Daily Fake I discussed the various types of historical sources and that the information merited a post of its own. Apologies for re-stating these basic points about history research that most probably know - but I think the principles are useful and should be at the core of any personal research if one hopes to approach truth for oneself. I hope this post will shine some light on what I think is the true metric to use when looking appraising evidence for oneself.

Background
One of the main problems with history is how to appraise the value of whatever historical artifact (literature, objects, architecture, etc). At school in the UK, I think I was taught 4 types. I saw this definition in a quora post, which I quite like:

1. Primary sources: These are original sources that provide firsthand accounts of an event, concept or discovery. Examples include original research papers, census data, interviews, letters, diaries, and photographs.
2. Secondary sources: These are sources that interpret, analyze or synthesize information from primary sources. Examples include books, reviews, and articles that provide a critical evaluation of research papers or a historical account of an event.
3. Tertiary sources: These are sources that provide an overview or synthesis of information from primary and secondary sources. Examples include encyclopedias, textbooks, and review articles.
4. Quaternary sources: These are sources that provide an overview of information on a specific topic, but do not present any new research. Examples include bibliographies, directories, and handbooks.

from: What are examples of primary, secondary, tertiary and quaternary sources? How can we identify them easily from each other while reading a... The special names are just 'numbers as text' first (primary), second (secondary), third (tertiary), fourth (quaternary).

Wikipedia also talks about these sources, but only has 3 types of sources - it drops the quaternary type.

In the study of history as an academic discipline, a primary source (also called an original source) is an artifact, document, diary, manuscript, autobiography, recording, or any other source of information that was created at the time under study. It serves as an original source of information about the topic. Similar definitions can be used in library science and other areas of scholarship, although different fields have somewhat different definitions.

from Primary source - Wikipedia

In scholarship, a secondary source[1][2] is a document or recording that relates or discusses information originally presented elsewhere. A secondary source contrasts with a primary source, which is an original source of the information being discussed; a primary source can be a person with direct knowledge of a situation or a document created by such a person.

Secondary source - Wikipedia

A tertiary source is an index or textual consolidation of already published primary and secondary sources[1] that does not provide additional interpretations or analysis of the sources.[2][3] Some tertiary sources can be used as an aid to find key (seminal) sources, key terms, general common knowledge[4] and established mainstream science on a topic. The exact definition of tertiary varies by academic field.

Tertiary source - Wikipedia

I think can see why wikipedia does not have quaternary - as a source it is not that different to tertiary - it seems that its only different in that it has 'panned out' even more; quaternary is that much further from whatever event is being discussed.

I want to point out that this sort of endeavour - talking about the value of a source - applies not just to history, but to almost any area of investigation where we are attempting to get close to the truth of the matter. So, science, law are also areas where this sort of source 'evidence scale' metric can also be applied. It is right that the evidence closest to an event is weighted most highly - primary source material (eg someone witnessing an event and writing about it at the time) should carry more weight than a tertiary opinion piece about the event in a newspaper. I hope this is all self-evident and unproblematic.

So, as a general scale to appraise evidence in various domains, we can see that this is a great methodology to help us to tread as solidly as possible.

The UN's UNISIST model of information dissemination
As I read through wikipedia's pages, I noticed that there was a reference to the "UNISIST model". I think this is interesting, because it is an attempt to codify an 'official metric' for evaluating evidence. This is what the UN does - it provides the detail and guides that one would expect from official sources.

UNISIST model - Wikipedia

My interest was piqued as the UNISIST wiki page has 4 sources, rather than the 3 wikipedia goes into. Instead of 'quaternary' we have 'Source literature' instead. Really, 'source literature' or, as they also call it "literature on the subject", is nothing more that official hearsay - it is the consensus opinion.

Suggestion for the individual when assimilating evidence.
I really have no strong objections to the 3 or 4 gradations of evidence. I think it absolutely essential to have some sort of means whereby one can accept this or that source as better than another. However, none of these sources are remotely comparable to the golden source. They are all flimsy hearsay in comparison to actual knowledge.

What is this "golden source" of which I speak? The golden source is personal experience. Personal experience is the only source for which one can say: "I know ..."

So, if you were there (at some event) and saw it close up - you have something far superior to even a 'primary source'. People often disparage 'anecdotal experience' when they say something like: "Its just anecdotal but ..." However, this is the most valuable information any individual can have. Disparage and dismiss your own experience at your peril: but if one doesn't accept one's own experience as relevant overall, how is it going to be possible for any individual to ever act autonomously? Not taking one's experience seriously, is to have already ceded authority to those who would define "the metrics/models" (eg the 'UNISIST model') and "the facts" (The Royal Society's Facts).

So if you were to ask me what the sources of information really are, I would give this ordered list:

1.	Present personal experience	it is unarguable that you know what you are presently experiencing	Unfalsifiable
2.	Historical personal experience, memories	memories can change perhaps, but you have unique access to your own memories	potentially falsifiable
3.	Primary sources	well-attested account	falsifiable
4.	Secondary sources	less well-attested account	falsifiable
5.	Tertiary sources	synthesis of primary or secondary accounts	falsifiable
6.	Quaternary sources / Source literature	consensus hearsay	falsifiable

One's present and historical personal experience are by far the best sources of information that one can get. This is experience should never be denigrated, even if it is not possible to share it or adequately express it to others. However, as we see above, when it comes to evidence, personal experience does not even get a mention! As we are never taught how to evaluate reality for ourselves as individuals, its not a surprise that humans mostly act on and believe in the provided sources, and frequently fail to act on personal experience/knowledge. We have each been educated into a lack of self-confidence in our own authority to guide ourselves but are taught to defer to government provided, licensed experts.

One cannot ignore personal experience in preference to an expert interpretation and hope to get close to the truth, when the closest one can get to the truth is by following one's personal life experience and interpretations.

 

[sekito]

Personally, I find personal experiences to be not much better than primary sources, because even if the experiences are ‘unfalsifiable’, people can easily be influenced to draw the wrong conclusions:

say, during the ‘COVID’ era, you got a ‘positive COVID test result’, and you feel sick, in fact ’sicker than the times you had the flu before’; and you know someone who had died ’because of covid’; an acquaintance of yours say they have ‘post-covid’ symptoms and feels permanently damaged - does that mean COVID is real and ’you have covid’?
or, say, your country is at ‘war’, and you heard sirens blasting in your city, you see burnt tanks on the street, buildings with charred and broken walls, alert alarms and messages being sent to your phone, and your acquaintances being drafted into the ‘army’ - does that mean the war is real, and there is actual fighting going on?

The situation is even worse with memories of experiences, as humans have the tendency to imagine past events based on their understanding/interpretation of the events - even if they no not make up new memorues entirely, vague memories will be biased towards a forgone conclusion: evidence agreeing with the conclusion are magnified, while evidence contradicting the conclusion is treated as coincidental, anomalous or erroneous.

*

To me, truth is not obtainable through direct means. It can only be guessed at through the process of eliminating what IS falsified as it is possible to prove if something is fake/wrong through contradiction.

 

[feralimal]

Thank you for your thoughtful response.

Personally, I find personal experiences to be not much better than primary sources, because even if the experiences are ‘unfalsifiable’, people can easily be influenced to draw the wrong conclusions

Right - how you frame whatever-it-is matters, as does how you interpret reality. But at least its your own experience. When you read a first hand account, this has to pass your framework - all info does - but you have deal with whatever story the author is trying to relay to you in addition to your (imperfect) interpretative framework. So direct personal experience is simpler as you don't have to account for whatever someone else said.

My broader point, is if you consider your own experience to have a greater value than 'provided information', you then have a chance of developing your own framework in a personally beneficial way. On the other hand, if you accept the 4 authorised sources of information only, and dismiss your own experience - you only allow yourself to work what you have been told.

The situation is even worse with memories of experiences, as humans have the tendency to imagine past events based on their understanding/interpretation of the events - even if they no not make up new memorues entirely, vague memories will be biased towards a forgone conclusion: evidence agreeing with the conclusion are magnified, while evidence contradicting the conclusion is treated as coincidental, anomalous or erroneous.

Yes - as above though, at least one is one's own intermediary of memories. Unlike say, watching historical shows on TV. So, this is still better than external information.

To me, truth is not obtainable through direct means. It can only be guessed at through the process of eliminating what IS falsified as it is possible to prove if something is fake/wrong through contradiction.

I don't think it is possible to obtain truth. I think one can align with it, be open to it, get closer to it. But in the first case, one needs to eliminate the nonsense, to even have a chance. And eliminating nonsense is not just dismissing what you were taught at school, its also fitting oneself into the reality one finds, as a first order participant with valid personal experiences, rather than dismissing oneself and one's experiences.