Geek warning. Non geeks use caution. Finding loops in ancestry trees is equivalent to a well known problem in graph theory, the problem of finding sets of strongly connected components in directed graphs. There is an efficient algorithm first described by Robert Tarjan, an American computer scientist, for finding the components. Each set of components is made up of elements forming a cycle; so in the genealogical application, a set of components makes up an impossible cycle of relationships.
I have incorporated Tarjan's algorithm in my Gedcom validation suite. Here is an example of the errors it catches. See the Gedcom file at the end of this post. It has four persons with an illegal loop, with Joan Hancock being both the mother of Thomas Wetmore IV and also a child of Thomas Wetmore IV. When run through the validator the following errors are reported:
Line Type Record
3 ancestryCycle 1 I1 Thomas Trask Wetmore IV
24 ancestryCycle 1 I4 Joan Marie Hancock
This is the shortest loop possible. If a loop consists of three or more persons every person in the loop is reported in the errors. If there are more than 1 loop, each loop is given its own index (here the index is 1).
0 @I1@ INDI
1 NAME Thomas Trask /Wetmore/ IV
1 SEX M
1 FAMS @F1@
1 FAMC @F2@
0 @I2@ INDI
1 NAME Luann Frances /Grenda/
1 SEX F
1 FAMS @F1@
0 @F1@ FAM
1 HUSB @I1@
1 WIFE @I2@
1 CHIL @I4@
0 @I3@ INDI
1 NAME Thomas Trask /Wetmore/ III
1 SEX M
1 FAMS @F2@
0 @I4@ INDI
1 NAME Joan Marie /Hancock/
1 SEX F
1 FAMS @F2@
1 FAMC @F1@
0 @F2@ FAM
1 HUSB @I3@
1 WIFE @I4@
1 CHIL @I1@