You've made a good start by testing at all three companies and uploading your results to GEDMatch -- as Judy G. Russell says in her recent post DNA testing for adoptees, "fish in all the ponds". The answer to "which test to take next" -- it depends. What do you want to know first?
Before you buy any product like this, it helps to know what information you can expect to get from the test results.
Russell's earlier post The Value of the Tests has charts on Y-DNA and mt-DNA inheritance that show which parts of your family tree might be revealed to you via DNA testing. FamilyTreeDNA's Learning Center has material geared toward beginners and more advanced researchers; the beginner's section has similar charts showing which parts of your lineage might be revealed, depending on which test you choose.
Hidden away at the bottom of the page is a link to FTDNA's recorded webinars, which explain using the Y-DNA and mt-DNA tests for genealogical matching and to explore deep ancestry.
The obvious advantage for doing a Y-DNA test is that it might get you a match with someone who is also participating in a surname study. Apart from that, it depends on how much each test will yield -- balancing how much information you want to gather with how much you can afford to spend.
My answer to the question Generational Loss of Data with DNA Testing? has a section of general resources, including links to many of the well-known bloggers who specialize in or write regularly about genetic genealogy. While you're waiting for other answers here, by reading the blogs and websites of the leading genetic genealogists, you should be able to find reviews and news about the specific tests that will help learn more. The more you understand what information might be revealed to you, the easier it will be to make an informed decision.
For AncestryDNA, there are also videos available that show suggestions about how to use their system by Crista Cowan, found on Ancestry's YouTube channel in their Desktop Education playlist. Crista demonstrates in a recent video how she builds out a tree on Ancestry when she is working with limited information:
- AncestryDNA: More Tips for Identifying Biological Family published on Jan 19, 2016
