FAQ page

Frequently Asked Questions

Some of the most frequently asked questions are answered here. If that does not solve your curiosity bug, be sure to contact us!

 

Does wScraper, Inc. utilize different technologies?

We use a range of the latest technologies. Our web scraping infrastructure was developed through Php/Python/NodeJS languages, which are high-level, Scripting programming languages. Our work over the years has included analyzing JavaScript/AJAX, using proxies, solving CAPTCHA’s, and working with and scraping millions of records. When engaged in large crawls, we’ve found Amazon’s ec2 cloud service to be superior and DigitalOcean’s Cloud as well. We are focused on utilizing the technology that works best for a specific task.

What languages do you speak?
We speak English for the purposes of conducting business.
Is scraping considered to be a legal activity?
The process of scraping data from publically accessed websites is commonly done and many large businesses, including Google, engage in the practice. Scraping is accepted and it is important to the survival and growth of a large number of online businesses. There is no problem with scraping as long as you reuse the data properly, and it is public information.
Is there a good way for me to learn about web scraping?
You’ll find some helpful web scraping information and resources on our blog. These are updated to keep you current and informed.
In what formats does wScraper, Inc. provide data?
We offer CSV format as our standard. We do this because analyzing with CSV is a straightforward process, and it is widely supported. However for the sake of client convenience and ease, we do offer alternative formats, including JSON, MySQL, NoSQL, Text, Excel Sheet etc.
Can you add more fields in the database than one would normally find?
We can include any field in your database as long as it is publically available. Simply request what fields you would like and wScraper, Inc. will include such if possible.
How much will it cost to scrape a website?

That depends on how complicated the process is. There are a range of factors that can influence the price. These include:

  • There may be restrictions on the number of page views per user, which translates into the need to use multiple IP addresses.
  • Data that is badly or inconsistently structured can extend the process.
  • Data that needs to be decoded can complicate scraping.
  • If Javascript is used to dynamically load, this can take more time.
  • Any data that is embedded in Flash or with images adds to the difficulty.
  • If there is a huge quantity of data, the job would take more time.

Prices vary and there are discounts for clients who order the scraping of multiple websites. Small sites with data embedded cleanly in the coding and that are structured in a clear and logical manner often cost around $100.

If you would like share your budge range, please let us know about this. We’ll be glad to review your requirements and consider the maximum budge for this certain project.

Is any website able to be scraped and data extracted?

Any website can be taken through the process of scraping as long as the data is publically accessible and available. However, even if information exists publically, it may beimpractical to try to scrape it. This occurs when a site puts major restrictions on IP addresses. When this happens it can take months to complete the job.

How long does it take to scrape a website?

Usually, simple websites take a few hours to scrape, however large sites may involve weeks of work. We will give you our best estimate regarding how long your project will take.

If I hire you for a custom scrape will you resell the data – Non-disclosure Agreement (NDA)?

No, we will not resell the data we have scraped for you through a customized process. We often deal with confidential and sensitive information. We are always ready to sign the NDA.

What is the process for hiring you?
It is simple. Just send us your requirements via Contact from, we will review it, and get back to you within 1 business day, informing you of the price and happy to answer any questions.
Once you are hired what is your process??

In each project, we follow a specific process designed to ensure that your job is done properly. The steps include:

  • We’ll discuss your project with you and find out what data you require.
  • We then crawl the website and download relevant webpages.
  • Using XPath or Regular Expressions,HTML DOMs we will extract the needed informationsuch as names, addresses, contact information from every webpage.
  • Document the information, placing it in an output file such as MySQL or CSV database.
  • Review results with the client and make any adjustments or changes as needed and prepare for updates.
  • Process final payment.
If I have a big project, can you handle it?
Yes, wScraper, Inc. can take on any size job, as we have a solid crew of professionals ready to get the work done.
Does my customized scraping include a source code?
Yes, it does. We utilize PHP for most projects. There are various websites that require the downloading of gigabytes of data. These are difficult to scrape without proxies. Data can be scraped again at a later date for an additional fee.
How does payment work?
 

If you decide to hire us after receiving a fixed quote for a custom website scrape and if you are a new customer, we’ll request a down payment of 50%. Larger projects may utilize milestones that denote what work will be completed at a specific time and how much is due once it that milestone is reached.

PayPal / Payoneer Card is our preferred method of payment, however there are other options available. If for some reason you are uncomfortable with making a down payment to a new company with which you have not worked before, we can set the job up on Elance/oDesk. This site has an escrow system that holds payments until a job is completed and approved. Due to the fact that Elance & oDesk charge a fee to freelancers, jobs that go through these sites will include that fee, which presently stands at Elance = 8.75% ‘n oDesk = 10%.

What is your policy on refunds?

Refunds are offered if for some reason wScraper, Inc. cannot complete the job or if you cancel the job before we have started to work on it. If for some reason you change job requirements after we have begun work, then we can renegotiate the fee.