The History of the DuWar Data Bases

The original databases of battles was developed by Trevor Dupuy and HERO (Historical Evaluation and Research Organization) back in the 1980s. They were published in a six volume work in 1983 as the HERO Land Warfare Data Base. This is back in the days when a data base did not have to be computerized (paper database – how quaint) and database was two words. It is report number 95 listed here: TDI – The Dupuy Institute Publications. Descriptive link is here: Analysis of Factors that have Influenced the Outcomes of Battles and Wars (dupuyinstitute.org). Of significance, there is a detailed description of each engagement in these paper reports. It was republished in 1984, 1985 and 1986 as report numbers 100, 103 and 111 here: TDI – The Dupuy Institute Publications. The final publication named the database as CHASE.

This effort was funded by CAA and was before my time. I came to work for HERO in 1987. There was then some back and forth between CAA, where HERO and CAA got to fighting over details of the content. One analyst at CAA sent 16 engagements out for comment. I did analyze that effort, although that file is now buried on an old Word Perfect DOS-era disk. He had four outside independent historians each analyze four engagements. The end result is the comments made corrections/improvements to 25% of the engagements, the comments did really did not change anything in 25% of the engagements, and the comments actually, if implemented, would have added error the engagements in 50% of the cases. This is fairly typical of outside comments, with 1-out-of-3 or 1-out-of-4 being helpful, and half of them would degrade the product. At that point, the project came to a griding halt, with much animosity between the arguing parties.

Then both HERO and CAA decided to independently computerize their databases. HERO added about four new engagements to their database, maybe corrected a few others, and the programmed it in a flat file called Reflex. It was 603 engagements (working off memory here) and called the LWDB (Land Warfare Data Base). CAA decided to computerize its version of 598 or 599 engagements and it was called the CHASE database. This became the CBD-90 that some people are still using. Neither of these versions included the extensive battle narratives as databases at that time could not handle large text files.

The computerized Reflex version of the LWDB was later purchased by Oak Ridge National Laboratories and published in the book by Dr. Dean Harley. It is a better version than the CBD-90. I did review the CBD-90 over twenty years ago. In the original database, there were a series of factors that were coded as to what degree they influenced the battle. In the CBD-90 about one-third of those factors (or one-third of the engagements that had those factors) – they were blanked out or mis-coded. It was a simple coding error, that as far as I know has never been corrected.

In the meantime, around 1995 I decided we needed to reorganize and reprogram the database. We had a new database created by Jay Karamales in Access. It included text files. We loaded the old Reflex engagements in the database and then Susan Rich and I proofed the entire database back to the paper copies. Susan Rich then entered in all the narratives into the database. So this was now a complete and proofed version of the 1986 paper database.

I then broke the database up. One of the problems with the original database is that it has engagements from 1600 next to engagements from 1973 next to a series of day-long division-level engagements from WWII next to some six-month long army-level engagements from the Great War next to battalion-level actions. While there are definitely some historical trends across all these, in some cases, depending on what you are analyzing, it is comparing apples to oranges. So, I took at mostly one-day battles from 1600-1900 and put them in a separate database (243 engagements – the BaDB. I took all the large army-level engagements (like Battle of Verdun, Battle of the Somme) and put them into a Large Action Data Base – LADB. Basically, moved them out of the way. They were later used in part to help create the CaDB (Campaign Data Base). I put the smaller battalion-sized engagements into a separate battalion-level data base (BLODB). They left us with a core of around 300 engagements in a division-level database, mostly of 1-day engagements. All this work was done outside and independent of any contracted effort and therefore became a Dupuy Institute proprietary product. As with any proprietary product, you have to protect it.

We then expanded all these databases. In the case of the division-level database (the DLEDB), we ended up doing a series of studies for CAA on Enemy Prisoner of War capture rates in 1998-2001. We coded the division-level engagements by outcome and then using that to analyze capture rates based upon the outcomes of the battle. This effort included getting counts of the number captured and the number of deserters in each engagement. This is reports E-1 to E-8 here: TDI – The Dupuy Institute Publications. The data used (but not the complete listing of the engagement) was included in appendices to these reports. CAA and the U.S. Army is still using these new rates.

We also added engagements to it from our urban warfare studies (CAA), reports U-1 to U-3. We used the database to analyze the urban versus non-urban combat. It was during that study we added engagements from the Channel Ports, Aachen and the three battles of Kharkov (1943). This study is discussed in two chapters in my book War by Numbers. We also took the time and put in 192 engagements from the Battle of Kursk (1943) based upon our work on the Kursk Data Base. All these Kursk engagements are listed (abbreviated) in my big Kusk: The Battle of Prokhorovka book. We also did a study on situational awareness for OSD Net Assessment (Andy Marshall’s old office). This is report SA-1 and also two chapters in my book War by Numbers. We ended up coding 295 division-level engagements based upon their knowledge of the enemy (by reviewing their intel reports of the divisions involved). We then reviewed what was the measurable combat advantage of improved situation awareness based upon real-world combat data. So, as in the EPW study, we took our original database and added additional filled-in fields so as to be able to do properly analyze the issue. This last expansion of the database was completed in 2004.

At that point, the division-level database had 752 cases in it. We had done some additional work on the old Italian Campaign engagements to clean them up and revise them. In particular Richard Anderson collected UK records from PRO and we cross-checked and revised all the UK engagements in the database and expanded the number of Italian Campaign engagements from about 70 to around 140. We then stopped work on the database in 2004.

During that time, we also expanded the battalion-level database to around 200 actions. We also had created a Campaign Data Base as part of our work, to examine operations above division-level and that last more than a few days. This was recently used for my presentation on Force Ratios that I gave at the second HAAC and in Norway in early November. See: The Schedule for the Second Historical Analysis Annual Conference (HAAC), 17 – 19 October 2023 | Mystics & Statistics (dupuyinstitute.org). In 2010 we created a small draft company-level database under contract with Boeing of 100 cases. A listing of most of these databases is here: TDI – The Dupuy Institute Publications. It does not include the company-level database, the Battle of Britain database nor the Dupuy Insurgency Spread Sheets (DISS) as we have not updated that page.

Obviously, people are going to ask: how can they get access to these databases. The answer is that you cannot until someone is willing to purchase them at a price that I willing to release them for. With the internet any single sale of the database will result in the release of the entire database to the world. So, any price would have to address the fact that these powerful and unique databases, which are proprietary to The Dupuy Institute, would be shared with the world. This includes potential business competitors. We still rely on contracts for our funding and these databases are part of our “product.” So, cost of giving away an exclusive competitive advantage? We would be willing to sell them to an organization if the price is right and they could then be publicly released. So far no one has made a significant concrete offer to us.

So other links:

Some Background on TDI Data Bases | Mystics & Statistics (dupuyinstitute.org)

Dupuy Institute Data Bases | Mystics & Statistics

Cost of Creating a Data Base | Mystics & Statistics (dupuyinstitute.org)

The Division Level Engagement Data Base (DLEDB) | Mystics & Statistics (dupuyinstitute.org)

Battalion and Company Level Data Bases | Mystics & Statistics (dupuyinstitute.org)

Other TDI Data Bases | Mystics & Statistics (dupuyinstitute.org)

Using the DLEDB:

Average Losses per Day in Division-level Engagements on the Eastern Front in 1943 | Mystics & Statistics (dupuyinstitute.org)

Density of Deployment in Ukraine | Mystics & Statistics (dupuyinstitute.org)

The U.S. Army Three-to-One Rule versus the 752 Case Division-level Data Base 1904-1991 | Mystics & Statistics (dupuyinstitute.org)

Comparing Force Ratios to Casualty Exchange Ratios | Mystics & Statistics (dupuyinstitute.org)