Forums, why did you have to go down after I made a huge answer to this ;-;
Basically, those robots are programs/machines that "work" for search engines (google, yahoo, bing etc) and they visit every page on the internet and save them. Saving them means they will show up when you look it up on google. Ex: I create a thread named 'Fish and more fish with Lily', until a "crawler" (another name for those robots) visits the page (which usually takes less than a minute), it won't show up on the google search. Once it has been saved, if I look up 'Fish and more fish', relevant searches come up, including the link to the thread I made.