About Us
OpenISearch is a creative center bringing together exciting and innovative ideas.
Our technology behind our new search engine will produce search results that are tailored specifically to what you are searching for. No need to search several times for the right results. Based on our proprietary indexing and result return method, OpenISearch will be able to bring the entire web to you, but also allow you to customize and control your results in real time!
We hope you love OpenISearch as much as we do.
About Robots.txt file
Robots are programs used to crawl the web and retrieve documents from websites. Robots.txt file controls and restricts access to Robots. In order to use the robots.txt, create a file named 'robots.txt' and place it in the root of your websites directory. To learn more about Robots.txt click here
Web Crawling
Search Engine related Robots, crawl the web in order to index information from websites. Each time our crawler (OpeniSearch) visits your website, it checks at the root directory of your server, for the robots.txt file. If it is found, OpenISearch bot will check for whatever rules or restrictions you've set in it. You can essentially customize the robots.txt to prevent and allow robots from accessing various parts or all of your site.
How can I restrict robots from indexing or crawling my website? Here is a list of various rules you can set in robots.txt file. You can set these rules by simply applying the appropriate lines outlined below into your robots.txt file you created in your root directory:
OpenISearch is a creative center bringing together exciting and innovative ideas.
Our technology behind our new search engine will produce search results that are tailored specifically to what you are searching for. No need to search several times for the right results. Based on our proprietary indexing and result return method, OpenISearch will be able to bring the entire web to you, but also allow you to customize and control your results in real time!
We hope you love OpenISearch as much as we do.
About Robots.txt file
Robots are programs used to crawl the web and retrieve documents from websites. Robots.txt file controls and restricts access to Robots. In order to use the robots.txt, create a file named 'robots.txt' and place it in the root of your websites directory. To learn more about Robots.txt click here
Web Crawling
Search Engine related Robots, crawl the web in order to index information from websites. Each time our crawler (OpeniSearch) visits your website, it checks at the root directory of your server, for the robots.txt file. If it is found, OpenISearch bot will check for whatever rules or restrictions you've set in it. You can essentially customize the robots.txt to prevent and allow robots from accessing various parts or all of your site.
How can I restrict robots from indexing or crawling my website? Here is a list of various rules you can set in robots.txt file. You can set these rules by simply applying the appropriate lines outlined below into your robots.txt file you created in your root directory:
| Allow all robots access to your site | User-agent: * Disallow: |
| Allow only OpenISearch access | User-agent: OpenISearch Disallow: User-agent: * Disallow: / |
| Block OpenISearch from Indexing your site | User-agent: OpenISearch Disallow: / |
| Block all robots from your site | User-agent: * Disallow: / |
| To Specify which files to Block or Access | User-agent: OpenISearch Disallow: /file_disallow Allow: /file_allow |
