Why Fallacies are False -- 08, Sampling Bias

Living in an echo chamber creates a fallacy called sampling bias.

It automatically excludes some true data from consideration.

You may ask “what if I have a huge circle of experts in my echo chamber?” You have to make sure your people really are experts.

There’s a legal definition for that. If the person has an extensive background of research and publishing in peer-reviewed periodicals, often with some teaching thrown in, that person can testify as an expert witness in court. But only in the field where they have done research. A psychiatrist can testify about a patient’s mental condition, but not about what caused a car crash involving the patient.

Or, if they have trained in a given field and worked there for years, they can testify as an expert witness. But only in the field where they have worked. An FAA controller can testify to how air traffic control works, but not about airplane engineering unless (see above).

People called as expert witnesses have been disqualified if they published only in periodicals that promote a specific line of thought. Their research is not scientific; the name for it is advocacy research. These people usually commit sampling bias; all their research is done among people in their echo chambers. A court will only accept their testimony if it is supported by work done outside the echo chamber.

Publishing books doesn’t count instead of or in addition to periodicals, unless the books are commercial versions of peer-reviewed professional publications, like the book form of an approved dissertation. Publishers are famous for getting authors to tart up their work to make it more exciting (I’ll say more about the excitement factor in later posts). Too many books get debunked; diet books all get debunked sooner or later.

Dr. A was an expert on biochemistry, but his book would have attracted an invitation from an echo chamber about the Bible, about which he knew nothing but what DH discussed.

Sampling bias and a number of other fallacies fail the data portion of the Test of Occam’s Razor, which says you have to cover all data that meets the description of your dataset, and do it honestly, without corruption or manipulation. I’ll talk about the manipulation part in later posts.

A number of fallacies have similar features to sampling bias; here are three of them.

The first is the one I talked about for the drug test. You pick a dataset and then you claim the drug is effective (a categorical) when your dataset only covers 10% of the world population. If you want a true claim that your drug is effective, your dataset has to be the start of a series of tests, each of which will address a different demographic. It has to be the “break up the problem and test one piece at a time” portion of Cartesian method. It cannot be the whole show.

Another is often called “cherry picking”. You do all the testing, then you report only on the successful trials. I saw this depicted on ER, once. A doctor was running trials of a drug, and reported out only the successful trials. The rest he grouped as “underlying unfavorable conditions”. A younger doctor assisting in the trials questioned this – and was fired. He was right to question it; he got fired over a bruised ego. Yeah, I know that was a TV show, if you know a lot about drug testing, speak up with a more realistic version and we’ll all learn something.

The third is one that I told Gary Curtis about, called The Texas Sharpshooter Fallacy. You create your output, then you create the description of the dataset to cover the outcomes you like. Sort of like, a guy shoots at his barn, then decides which ones he wants to brag about. Draws a line around them and claims that was the target all along.

A partially related fallacy is “quoting out of context”, which I discuss on my blog in three posts. 

Quoting out of context has been used for millennia to influence people to think or behave a given way.

Sampling bias also relates to weak analogies. An analogy might ignore inconvenient truths in the interests of making a point. An archaeologist once claimed that an inscription referencing Balaam was a reference to the Balaam in the Bible. It was a Moabitic site, which coordinated with Balaam in the Bible working for the king of Moab. But Balaam was part of the Exodus story, and the Israelites existed in the Holy Land in time for Merneptah to write his stele (1230s BCE), while the Moabite inscription dated to the 800s BCE. Add to that the 100% difference between what the inscription and the Bible record, and that archaeologist proved nothing except their own incompetence.

There’s a tradeoff between dataset creation and the use you can make of it. But remember, you’re the one who decides on the dataset, and unless you cast a broad enough net, your claims of “I haven’t seen…” become irrelevant.

Why Fallacies are False -- 07, Categoricals and Negatives

The phrase “I haven’t seen…” is an attempt to assert a categorical negative claim. Let’s look at categoricals.

A categorical claim says that all X are Y.

If you can find even one example of X that is not Y, the statement is false.

And since none of us are omniscient, nor can we access every piece of evidence that might exist, we cannot make categorical claims – UNLESS WE HAVE A COMPLETE DATASET.

To get a complete dataset to make claims about, we need a clear definition of what is in our dataset and what is not. But we can make the definition so stringent we defeat ourselves.

Real world example? You want to test a drug. If your dataset of test subjects is strictly white males between 20 and 50, your results don’t necessarily apply to women, to children, to the elderly, or to POC. So you can’t make the categorical statement that “Drug X is 75% effective” except for test subjects who represent 10% or less of the world’s demographics. But decades ago, everybody assumed that if it worked on white males of a given age, it would work on everybody else. People died over that assumption.

The “all X are Y” claim means every member of one dataset is also a member of another dataset or has the same description. The problem comes if you claim that all X are not Y. This can mean two things.

One is that no member of dataset X is in dataset Y or matches the description in dataset Y. When we’re just talking normally, saying “all X are not Y” could mean that some X ARE Y: “all dogs are not vicious” in normal conversation could mean you’ve been talking about dogs that are vicious but you want to point out that viciousness is not a universal quality of dogs.

That doesn’t work in logic. If you mean “some are not”, you have to say “some are not”.

The other thing it can mean is that you are confusing inherent qualities (essence) and qualities perceived or caused (accidence). This is the basis for the “all dogs are not vicious”. The person saying that probably knows that historically, abused animals turn mean. They’re saying that dogs are not inherently vicious, the viciousness is caused.

Negation is an operation in Boolean algebra. It is a categorical claim that something does not exist. When you don’t have a complete dataset, making a negative claim risks somebody finding the exception that proves your claim doesn’t hold water.

“I haven’t seen…” is an attempt to pretend that you have a complete dataset. In reality, on social media for example, it only means that you don’t follow every account on every possible platform.

Instead, it often means that you only follow the accounts of people who already agree with you. It’s called “being in an echo chamber”. If you want to say “I haven’t seen…”, make sure you do it inside your echo chamber, because people outside it may have the evidence that what you say is false. And so I have a habit of replying “follow more accounts”, especially when I HAVE seen whatever the OP is denying.

Instead of saying “I haven’t seen…”, give the hard evidence that the probability of existence is infinitesimal. Calculating an infinitesimal probability of truth shows that an argument is “down in the noise”, not worth troubling your head over. That’s what forensic DNA testing does, tells the court that there’s an infinitesimal probability that somebody other than the accused provided the sample that gave the test results.

Now. In logic, you can also defeat a negative by proving that it creates an absurdity, reductio ad absurdam. It’s kind of like what Elle did to her ex in class in Legally Blonde. He made a categorical claim that a guy was not stalking the girl who birthed his child, he was worried about what happened to the sperm he had donated. Elle reduced that to an absurdity by asking if he would do the same thing if he donated to a sperm bank or had a nocturnal emission. But that was in class, not in court.

So let’s be careful about both the categoricals and the negatives that we try to prove.

Knitting -- videos

Time has passed since I started my Knitting thread. Some of the videos I link to don't work any more. 

Johnny Vasquez' New Stitch a Day site used to have videos for basic stitch patterns but they have reorganized on some stupid principle you will never figure out and a search for "basic knit stitch" doesn't turn up a video.

Here's one solution. I have used a number of Joanne's patterns.

https://joannesweb.com/how-to-knit-the-absolute-beginners-guide/

This page has a video for the long tail cast on that I have learned to love, as well as knit and purl stitches.

https://joannesweb.com/how-to-knit-the-absolute-beginners-guide/

If you've been using my knitting blog and got frustrated by links that didn't work, I hope this helps.

Why Fallacies are False -- 06, Argument from Silence

The issue of hard evidence is important to every argument. When you don’t have any hard evidence, you’re done. You can’t prove anything. Right? Right?

Have you ever heard the phrase “the absence of evidence is not evidence of absence”?

I’ll give you a concrete example some of you may be too young to know about.

There are currently no Buddhist statues in Afghanistan. Does that mean there never were?

Us old people may remember the Bamian Buddhist statues carved into rock in Afghanistan. The Taliban blew them up. Pictures remain, but if you don’t know about the history or you’ve never seen the pictures, you might claim those statues never existed.

This is a fallacy called “The false argument from silence”. Does that sound familiar?

There is a true argument from silence. You can’t make it unless you have the complete dataset.

So if you claim that HBO is not running a specific film, all you have to do is show the HBO program schedule. If that film isn’t in the schedule, it follows as the night the day that HBO is not running it. HBO has complete control of the dataset and they are not going to list something they are not running.

But when anybody goes on social media and says “I haven’t seen…” they are running into the false argument from silence for several reasons.

First, they are not Gd. They are not omniscient. There can be many things they do not know. I have labeled this The Omniscience Fallacy but it’s a subset of the false argument from silence.

Second, the world churns out 1 terabyte of data daily. It is not possible for a mortal to access every piece of data the world turns out.

“I haven’t seen” is also a bigger issue called making a negative claim, and it’s related to categorical statements, which deserve a post of their own.

But on the subject of the false argument from silence, before there was a terabyte of data in the world, the false argument had concrete examples. If you’re a paleontologist, you know it as natura non facit saltus.

No paleontologist can discover a new fossil without knowing that at least half a billion years of ancestors lie behind it. That’s because the hard evidence for the age of earth says it is 4.5 billion years old, and the hard evidence for the history of life says it goes back 4 billion years. There’s no such thing as finding a fossil, and thinking that it had no parents, grandparents, or other antecedents. You don’t need hard evidence of all the antecedents to know they existed.

And we know of many mechanisms that deprive us of the remains of living things: fires; volcanoes; earthquakes that bury them; tectonic plate shifts; normal decay and weathering; rockslides and cave collapses.

On my blog, I change this for archaeologists: cultura non facit saltus. There are no archaeological remains that arise out of nothing. They all have cultural antecedents. But a 350 century old archaeological site had 1 micrometer per year of remains, in a hunting camp repeatedly occupied across human evolution, by both Neanderthals and Cro Magnon. As time went on and human tools became more durable, remains at the various archaeological sites became deeper; ceramics lasted longer than leather bottles and metal survived longer than ceramics, except for precious metals and iron which were remelted and recast.

And any historian who talks about Dark Ages and pretends that the next stage in a culture arose out of nothing can go pound sand, for the same reason. No historian has a complete dataset even for a single year in time, let alone for the antecedents of whatever they are studying.

So with my Torah example, claim one saying that there is a single source for it, does not need hard evidence for support. Torah definitely exists, and being a cultural artifact, has a history behind it involving that culture. We will never have all the steps in that history. But denying that there is a history is bad logic. I’ll say more about this at the end of this thread.

The same is true for DH. Just because we have no hard evidence for JEDP, doesn’t mean they didn’t exist. The problem for DH is that the concept is full of false data and fallacies.

So next time you are tempted to say “there’s no evidence of that”, stop yourself and ask “do I have a complete dataset?” Being that we are not omniscient and can’t cope with all the evidence that might exist, the answer is probably no. Even if you’re a specialist in the field, remember my professor, who wrote the dissertation on Biblical Hebrew and didn’t know that there was external evidence that what he wrote about was a thing.

I’ll say more about incomplete datasets later, but next week I’m going to clean up after myself.

Why Fallacies are False -- 05, Conjunctions and Documentary Hypothesis

Here’s why I love Linda. A quick review.

The Linda problem starts with a dataset describing Linda. You can say anything you want about her. It doesn’t even have to be true.

Then you try to decide which of two statements is more likely to be true:

1.     Linda is X.

2.     Linda is X and Y.

There is no relationship between X and your dataset about Linda. There is also no relationship between Y and your dataset, or between Y and X. So you can’t prove that either X or Y is true about Linda based on the dataset.

To turn this into math, assign a probability of truth to each of X and Y, and the number is between zero and one because you can’t prove anything about either one. Because statement #2 is a conjunction, the math is multiplication. When you multiply two fractions smaller than 1, the product is always smaller than both of them. That means it will be smaller than X. So the product of #2 is smaller than #1 and 1 is more likely to be true. But most people get that wrong the first time they meet a Linda problem, and even after you explain it, some people still get it wrong the next time they meet one.

Linda’s formal name is the Conjunction Fallacy.

Here’s why I love Linda. The Documentary Hypothesis has a dataset describing at least four putative documents. DH claims that my Torah is made up of these four documents. You may have heard of JEDP; that’s them.

DH denies that “Linda is X” is possible. They reject any description of “Linda” that allows the simple statement.

DH insists that “Linda is Y and Z and…”, that is, Torah is made up of four sources. The probability of that is the product of the probability that each assignment to one of the four putative documents is correct. There is no hard evidence that any of them existed, so the probability of every assignment is between zero and one.

To get the answer, you must multiply because DH says you can’t assign the whole thing to just one document, and you must have enough assignments to achieve the whole Torah. So the answer is some fractions between zero and one multiplied together, with at least four terms contributing to the product.

Now, it would be one thing if DH took each book in Torah and assigned the whole book to one of the four sources. So your terms might be J for Genesis, E for Exodus, P for Leviticus, D for Deuteronomy, and then one of them again for Numbers. 

But the dataset doesn’t support that because the descriptions of the documents don’t match any one book. P gets the closest to Leviticus and assignments to P would have the highest value – if they were all restricted to Leviticus.

But they aren’t. DH splits all five books. Numbers is split up the most, with parts from each of the four documents. Leviticus is split up the least, but some of its putative sources are not even JEDP. But at any rate, the number of terms is larger than four and you might as well make it ten for starters. That would be the outcome if Leviticus was all P and Deuteronomy was all D, and each of the other three books was split between two or three of the documents.

But they’re not. All right, then, it would be nice if DH took each narrative in Torah and said it came from one of the four sources. That means you’re looking at each narrative as one term and assigning it a probability of coming from one of those four documents. I’ve counted some 80 narratives in Torah, stories with plots and characters and action (and this feeds into something I will talk about later). So there are at least 80 terms in the calculation, all of them fractions between zero and one. And then you have to consider the non-narrative portions, which fall between the narratives. So the number of terms is larger than 80; we could set it at 100 for starters. A fraction between zero and one to the 100th power is infinitesimal, down around 10 to the minus 61. (For comparison, the Planck length is 10 to the minus 35 meters.)

But that’s not what DH does. DH splits some narratives up and assigns part to one document and part to another. So you can’t count on two verses that are sequential, being assigned to the same document. Actually, it’s worse than that, because DH splits some verses up, assigning part of the words to one document and part to another. But let’s ignore that last bit, because what I’m saying is that DH has a probability that is the product of some fractions between zero and one. You have the same number of terms as the number of pieces in the DH assignment, something more than 100 terms.

Since you can’t count on a chunk of verses to all be assigned to the same document, you have to consider every verse a separate term. 

Torah has 5845 verses.

DH’s probability calculation has at least 5845 terms. Each term has a value between zero and one. If every term had the same value, the answer would be that value raised to the 5845th power. The answer is infinitesimal. The probability of DH being true is vanishingly small.

Now, DH will say that it is not a Linda problem, there IS a connection between the dataset and the assignments, the dataset describes the four documents to which they are making the assignments. But as I said, we don’t have hard evidence (yet) that they ever existed. What’s worse, I show on my blog that the descriptions themselves have a basis in fallacies, two of which I will get to later. Worst of all, I show that, from the start, DH relied on false data. This is what I wrote about last time: even if you don’t have a real conjunction fallacy, but your dataset contains falsehoods, you’re wrong. The whole concept has a zero probability of being true.

My science author was a trained biochemist but, like I said last week, that doesn’t mean he had training in logic. And even if he did, it's obvious that he didn't subject DH to a probability calculation. He had no training in the Bible, he admits that. It was one of a number of instances where a scientist writes about something they’ve never researched, and the work gets attention because of who they are, not because they know what they’re talking about. I won’t go into that rant here.

Knitting -- ripple throw with piped edging from leftovers

I haven't posted on this page in a long time because I was working on a project which I just finished.

A long time ago, when I was in high school, there was this craft thing where you had a wooden spool with a large hole down through it. At one end there were some thin nails, and you used a crochet hook to loop yarn around them and then sort of knit a long snake. 

Technically this is called an I-cord. You can use it for the edge of a coat; I used to have a coat pattern with this specific design.

Otherwise the only other thing you might do with an I-cord is make a piped edging for housewares, like a cushion cover. So let's do that.

I worked this project on straight needles because my circular needles already had projects on them. The leftovers were Comfy fingering and I used size 3 needles.

This being a throw, it was rectangular. I used Arne and Carlos' video to make the I-cord for the bottom edge, but I used 3 stitches not 4.

https://www.youtube.com/watch?v=T_lQU5QsdNs

I made an I-cord that was 312 stitches long. This gave me 306 stitches for the pattern and 3 stitches on each end for the side I-cords.

I used a crochet hook to pick up the Vs on the top of the I-cord to work the yarn into them, then I knitted the next row with I-cord stitches at each end.

ON RIGHTSIDE ROWS:  K1, yarn forward, slip purlwise, yarn back, K1. Work across in pattern, then K1, yarn forward, slip purlwise, yarn back, K1.

ON WRONGSIDE ROWS: slip 1 purlwise, yarn back, K1, yarn forward, slip purlwise. work across, and then yarn forward, slip purlwise, yarn back, knit, yarn forward and slip.

The pattern is an 18-stitch repeat. Work this between the 3-stitch edges. If you want to work it stand-alone, K1 at the edges on R3 and R4.

R1 & 2: Knit  

R3: *[(Knit 2 sts together) x 3, Knit in front and back of next stitch, (Yarn Over, K1) x 4, K in front and back of next stitch, (Knit 2 sts together) x 3], repeat * till last stitch.

R4: Purl to last stitch.

Work this until all your leftovers are gone -- or at least until there's not enough of the leftovers to do a full 4 row pattern. I used 4 sets of the pattern for white and 5 sets for all the other stripes. One skein of yarn made three sets of 5-pattern runs. I had one full skein of that dark pink and it made all three of those stripes.

When your leftovers are used up, add in the color for the top piping. I recommend working the same  number of repeats at the top as you did after starting the bottom I-cord.

For the top bind-off work another series of piping.

Put a DP needle into the three stitches of the edge piping and pick up the first stitch of the top row. Using another DP, K1, yarn forward, slip purlwise, yarn back, K2TOGTBL leaving the last stitch on the needle. Now pick up the loop in front of the next stitch and do this again. Pick up that same loop and K2 K2TOGTBL. This turns your corner.

Use one of your DPs to * pick up the next stitch of the top row, K2 K2TOGTBL leaving the last stitch on the needle * until all the top row stitches have been worked. Slip two stitches over to make an edge for the piping and work the end of the yarn into the wrong side. Notice that this will roll the I-cord just as on the bottom since you never turn the work over and purl.

If you want to work this in bulky, make a bottom I-cord 132 stitches long.

This I-cord piping makes another nice non-curling edge and, as you see, it follows the ripple in the pattern. 

Why Fallacies are False -- 004, Conjunctions and Conspiracy Theories

You’ve had a week now to think about that last statement. If you teach somebody about the Conjunction fallacy and why it’s wrong, at a later time they will STILL PICK THE WRONG STATEMENT AS MORE LIKELY.

If they’re looking at the exact same problem, that’s nuts. But it could be a matter of not realizing they’ve seen it before. We all forget things.

To show you how conspiracy theories are Linda problems, I have to emphasize some things.

First, it is irrelevant whether the “description of Linda” dataset is true or false. It’s just a bunch of information labeled Linda for the purposes of the explanation. You could collect sports statistics and that would still be your dataset.

The important thing is that both your X and your Y have nothing in common with that dataset. In the classic problem, the first statement is that Linda is a bank teller. Nothing in the description says she went to business school or college or any training sessions that qualify you to be a bank teller. If your conspiracy theory is a real conjunction fallacy, the simple statement will have nothing in common with your dataset. If it does, you’re not looking at a conjunction fallacy.

The same is true about your Y, which in the classic problem says Linda is a feminist. Again, there’s nothing in the dataset that says she belongs to NOW or any other woman-oriented organization, or that she participated in feminist protests. Again, if the “Y” in your conspiracy theory actually has a relationship to your dataset, it’s not a real conjunction fallacy.

At that point, it becomes important if your dataset has falsehoods in it. If you can prove that, you prove the whole thing is probably not true. Same if the connection between the dataset and the other statements have fallacies in them.

Once you pick your dataset, be careful with the claims you make about it. Prove the X claim. Then you can go on and prove the Y claim. And THEN you can make a probably true conjunctive claim involving X and Y. And this is exactly what science does, under Cartesian method: break a problem up into pieces; prove one piece true at a time; and then combine them.

Highly educated people face as much risk of creating and believing conspiracy theories as people who left school in their teens. Higher education does not force people to learn about fallacies. Even if STEM training teaches Cartesian method, they don’t necessarily lecture on the connections to logic or why the Method helps prevent creation of fallacies.

Next time I’ll give an example of a conjunction that never heard of Cartesian method.