As you have discovered, it can be tricky to find a person's origin if you only have one document. Your first step should be to find more documents in the US to see if you can get more information. Different documents might give you other clues, either a different spelling of the same name, a different jurisdiction, or a name of a nearby town. When you correlate all available data, it is easier to see which place was meant.
Use gazetteers (dictionaries of place names) and see if you can match up the information with a known place. Choose a gazetteer closest in time to the document you are looking at, and keep in mind that documents may reflect the name the place had at the time the document was created (e.g. some documents will say Russia where others say Poland).
Other cautions: Documents were likely to have been filled out by a clerk, and that person may not have known the place. Passengers buying tickets with steamship companies filled out an intake form which was then copied onto the big sheets created to collect information for immigration. Any time information is copied, typos can creep in, and the passengers wouldn't have seen the manifests and couldn't have pointed out errors. "Correct spelling" of words, including place names and surnames, is a late-20th century idea; spelling was fluid even in the early 20th-century.
Also, people from very small places are likely, later on in life, to have given the name of a larger town close to where they came from that was better-known. Pay particular attention to the name of the informant on the death certificate. A child may only know the name of the nearby large place, not the name of the original small place -- usually a disadvantage, but for you, it could be a valuable clue to finding a smaller place.
Search for:
- A Declaration of Intention (sometimes called "first papers)
- Passenger lists
- Passport Applications
- Newspaper articles
- Death Certificates
- any record likely to list a birthplace or religious affiliation in the old country (see the record finder in the resources list)
If your great-grandfather had siblings, look for their records. Examine the entire family as a group. On passenger lists, pay particular attention to the names of the closest relative in the old country. Where does that person live?
Take a look at the other questions and answers on the site for identifying places in Poland that were at some point part of Russia. You can see what strategies and resources were used in the answers and see if those can help you.
Note especially this comment from this answer to Where would this town/area be located in Poland, today?.
Please also bear in mind that 100 years is enough for small villages
to completely disappear, making the search even more difficult. –
skolima
Resources
Articles from the FamilySearch Research Wiki
FamilySearch Lessons have video and handouts available for download.