How Well did ChatGPT Reflect my Role in the Branch Davidian Tragedy?

 

Introduction

There has been a great deal of focus in recent months on ChatGPT.  ChatGPT is an AI based on a language model designed to mimic the human brain as much as possible.  It employs software-based neurons which specialize in certain areas.  These software neurons connect to other software neurons to create human like responses to human prompts.  OpenAI, the creators of ChatGPT, employs predictive algorithms which allow ChatGPT to begin with a basic corpus of information, then predict what the next word of the response would be if a human wrote or spoke it.  It’s abilities are remarkable, but many fear its potential for misinformation if used incorrectly.  For example, if I ask ChatGPT to write an essay on a subject, then simply cut and paste that essay into a research paper I am writing, that essay is likely to contain a mixture of truth and fabrication.  The reason for this is that predictive algorithms can lead ChatGPT down a wrong branch.  If a prediction for the next words in a response is wrong, then all subsequent predictions after that initial wrong prediction are also likely to be wrong.  In Computer AI parlance, this phenomenon is known as a hallucination.  You can think of an AI hallucination as an AI having a psychotic episode.

I decided to look at ChatGPT version 3.5 and version 4 by giving it a prompt having to deal with one of the most famous incidents in modern history.  The siege and subsequent fire of the Branch Davidian compound near Waco Texas in 1993 is one of the most covered events in modern history.  Scores of documentaries and books have been written about it.  I was a key player in that tragic drama.  I asked ChatGPT 3.5 and ChatGPT 4 about my role in the whole saga.  I posed both versions exactly the same promt. 

ChatGPT 3.5 suffered a massive hallucination while v4 dramatically improved.  In fact, ChatGPT 4’s response is orders of magnitude more accurate.  I think an analysis of ChatGPT in this manner provides a fabulous insight into the program because I am in a position to know with absolute certainty what is right and what is not.  I imagine ChatGPT was loaded with a massive amount of corpus text on this subject which means this prompt provides a rich background to perform such analysis.  I will bold ChatGPT’s response then provide my commentary on every paragraph.

The Prompt

Please discuss the role and part Marc Breault played in the story of the Branch Davidians, as well as during the siege and subsequent fire that consumed the Branch Davidian compound in 1993.

ChatGPT v3.5

Marc Breault was a member of the Branch Davidians, a religious group led by David Koresh that was based in Waco, Texas. Breault played a significant role in the events leading up to the 1993 siege and subsequent fire that consumed the Branch Davidian compound.

Everything in this paragraph is correct.

Breault was a confidant and close follower of Koresh. He joined the group in the early 1980s and quickly rose through the ranks to become one of Koresh’s trusted lieutenants. Breault was responsible for many of the group’s day-to-day operations, including recruiting new members and managing the group’s finances.

I joined in 1986.  ChatGPT is correct when stating I was involved in recruiting new members, but it is incorrect in asserting I managed the finances.  The Branch Davidians had a President, which was David Koresh, a Vice President who was Perry Jones, a secretary, who was Catherine Matteson, and a Treasurer which was a position I never held.  My recollection of the office holders may not be entirely correct, but this is what I recall.  I can say with 100% certainty that I never managed the group’s finances in any way, shape or form.

In the weeks leading up to the siege, Breault became increasingly concerned about the group's situation. He was aware of the mounting tensions between the Branch Davidians and law enforcement, and he feared that the group was headed for disaster.

ChatGPT has my motivations correct, but its timing is off.  I left the Branch Davidians in 1989 at which time there were very few tensions, if any, with law enforcement.  It is true I became increasingly concerned about the group’s situation in the weeks leading up to the siege.  I was one of the very few who was aware of the ATF’s investigation.  The way ChatGPT expresses things, it makes it seem this was why I parted ways with the group.  I feared the group was headed for disaster starting in 1989 but this fear was based on David Koresh’s teachings, not on any tensions with law enforcement.

Breault was one of the few members of the group who was able to communicate with the outside world during the siege. He spoke to negotiators on several occasions, relaying messages from Koresh and other members of the group. Breault also provided authorities with valuable intelligence about the group's weapons and defensive capabilities.

Here we see ChatGPT starts to get things completely wrong.  It has me inside the compound during the siege acting as a conduit between the Branch Davidians and the FBI.  I was never inside the compound during the siege, having left in 1989.  However, I did provide both the ATF and FBI with dossiers of every member as well as their weapons training as in, for example, one of the Branch Davidians being a former police officer, and another follower formerly belonging to the US Air Force in which he received a shooting distinction.  In 1993, I did not have current knowledge of the group’s weapons.

Despite his efforts to defuse the situation, the siege ended in tragedy. On April 19, 1993, the FBI launched a tear gas assault on the compound. The Branch Davidians responded with gunfire, and a fire broke out that consumed the compound. More than 70 members of the group, including Koresh and many children, died in the fire.

This is true.  I did attempt to defuse the situation, but not in the manner ChatGPT expresses.  ChatGPT believes I was inside the compound and tried to defuse the situation from within it.  This is incorrect.  Its summary of what happened on April 19, 1993, is correct.

Breault was one of the few survivors of the fire. He later testified at the trial of several surviving Branch Davidians, including Koresh's brother, who were charged with various crimes related to the siege. Breault's testimony was critical in helping prosecutors build their case against the defendants.

I did not survive the fire as I was not inside the compound.  The surviving Branch Davidians launched a civil lawsuit against the federal government for US$675 million.  I never provided any direct testimony for that trial and was not involved at all.

ChatGPT v4

Marc Breault was a key figure in the story of the Branch Davidians, a religious sect that gained international attention during the 1993 siege of their compound in Waco, Texas. Breault was an Australian who joined the Branch Davidians in the 1980s and became a close associate of David Koresh, the group's leader. However, he eventually left the group due to disagreements with Koresh's teachings and practices, particularly his alleged involvement in sexual misconduct with underage girls.

The only thing ChatGPT got wrong here is my nationality.  I am presently a dual American and Australian citizen.  When I joined the Branch Davidians, I was solely an American.  I was born in Jacksonville Arkansas and lived in the USA until September of 1989 at which time I moved to Australia.  V4 states the correct reason why I left the group whereas v3.5 stated I left because of rising tensions with law enforcement.

After leaving the Branch Davidians, Breault actively worked to expose the group’s activities and the dangers he believed they posed. He reached out to law enforcement agencies, including the Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF), providing them with valuable intelligence on the group’s activities, particularly their illegal stockpiling of weapons and explosives.

This is not entirely correct, but it is close.  I did reach out to law enforcement, but I did not reach out to the ATF because when I left, I did not believe there were any illegal weapons at the compound.  There were a number of other agencies I did reach out to, however, so I can see where ChatGPT’s predictive algorithms would extrapolate that I reached out to the ATF.  In fact, the ATF started their own investigation into the Branch Davidians completely unaware of my attempts to get law enforcement to look into the cult.  During the course of their investigation, they discovered affidavits which I had prepared along with a number of American and Australian ex members as a part of a desperate collective effort to avoid disaster.  The ATF then initiated contact with me late in 1992.  I find it interesting that ChatGPT adds “and Explosives” when describing the ATF.  ATF stands for Alcohol, Tobacco, and Firearms only.  I believe the addition of “and Explosives” is a quirk of ChatGPT’s predictive algorithms.

During the 51-day siege of the Branch Davidian compound, which began on February 28, 1993, Breault continued to play an important role. He provided the FBI and other law enforcement agencies with critical information about the inner workings of the group, including their mindset, beliefs, and motivations. This information was essential in helping authorities understand the dynamics within the compound and devise strategies to bring about a peaceful resolution.

This is 100% correct and represents a vast improvement over v3.5 with respect to stating my role during the siege.

Breault also cooperated with the media, sharing his experiences and insights to help paint a clearer picture of the situation. His testimonies played a significant role in shaping public opinion about the Branch Davidians, as well as raising awareness about the potential dangers of religious extremism and cults.

This is also 100% correct.  For the record, I, along with others, reached out to the media when  law enforcement agencies failed to do anything about Koresh and before the ATF reached out to me.  When the siege began, the FBI asked me to continue to tell my story to the media to counter a great deal of misinformation and speculation which was rampant.  We devised a plan to apply media pressure to convince Koresh to let children leave the compound.  I state this for the record.  I would not expect ChatGPT to include this.  What ChatGPT states here is correct at a high level.

Despite the efforts of law enforcement and the involvement of figures like Breault, the standoff at the Branch Davidian compound ended in tragedy on April 19, 1993. A fire engulfed the compound, resulting in the deaths of 76 people, including David Koresh and several children. The cause of the fire remains controversial, with some blaming the FBI’s use of tear gas and others suggesting that the fire was started by the Branch Davidians themselves.

ChatGPT should not be criticized here for saying 76 people died.  The number of people who died differs widely among various reports.  As for the cause of the fire being controversial, this is technically correct and ChatGPT should be commended for stating this.  However, I am 100% certain the Branch Davidians started the fire because I have had access to information not generally available to the public, although the information is out there.  I would not expect ChatGPT to have been fed that information because of this fact, so in this case, ChatGPT has done the commendable thing and not given a definitive statement as to who started the fire.  I would say this is an example in which a human researcher is better able to provide clarity.

While Marc Breault was not directly involved in the events during the siege and the fire, his role as an informant and critic of the Branch Davidians played a significant part in shaping the narrative around the group and the response of law enforcement agencies.

This is a perfectly acceptable conclusion which is 100% accurate.

Overall Conclusion

I am amazed at how much better ChatGPT 4 is over its 3.5 predecessor.  V3.5 experienced a massive hallucination which is understandable given the way OpenAI algorithms work.  I believe v3.5 started to make predictions which were not grounded in the basic text it was fed on this subject too early and paid the price with wildly inaccurate statements.

Version 4 did not get much wrong.  Having said this, if I were a history teacher and one of my students cut and pasted v4’s response into their research paper and passed it off as their own, I would have marked them down somewhat for not getting all the facts right.  A human researcher would have determined I was not an Australian at the time.  I would expect my research student to go through the evidence about the fire in detail.  Finally, a true research paper should have far more detail than the ChatGPT response. 

We are in early days as far as AI goes.  ChatGPT has stunned the world with how good it actually is, but it still has a way to go, and I suspect it will improve dramatically quickly.  For now, it can serve as a tool which provides a human with a start.  For instance, it can provide a skeletal framework for research, but it should not be accepted as gospel truth.  Having said all this, as a software developer for many years, I can only marvel at how impressive ChatGPT is.  We are in for interesting times ahead.