Turn websites into you can use

Who are we?

ThunderMetrics is the data services division handling data mining, crawling & analytics services of ThunderQuote(TQ), a B2B platform specialising in creating intelligent eProcurement systems.

ThunderMetrics also powers TQ’s intelligent corporate due dilligence and vendor sourcing & matching algorithms through machine intelligence.

Our data pipelines power sales, marketing & research functions globally

Web data, without the fuss

Our web data scraping & web data mining solutions have helped businesses all the way from small businesses to F100 multinational corporations. Just tell us what kind you need, and from where, and we’ll take it from there.

Share your project requirements with us so we can generate a quote for you, pricing is based on the websites scraped, amount of data, and complexity of sites scraped. 

Why ThunderMetrics

Experts in Web Scraping Technologies

Ultra-Fine Web Data Scraping,
At Mass Scale

Zero Customer  Lock-In

Intelligent Semantic Keyword Detection

Data Or Your Money Back Guarantee

How are you using web data in your business?

We provide web data scraping & mining services which can be used to accomplish the following use cases and more:

1

Real Time Price Intelligence

 
Get an edge over your competitors in pricing; position your products; make the right pricing decisions with pricing data extracted from ecommerce platforms and sites

HR & Recruitment Intelligence

 
    Find out who is recruiting and for what roles, and engage them for sales purposes with the freshest job listing data 

Targeted Lead Generation

 
 Generate smart targeted lead lists from directories of customers, through web data scraping

Social Listening & Brand Monitoring

Protect your brand; analyse consumer behaviour, track market sentiment and reputation on the social web, in real time
 

Market Research

 
Entering a new market? Scan entire geographies, territories for your target customer segments, know who they are, how large, and how to get in touch 

Machine Learning Training Data

Programmatically generate labelled training data from the web for your AI / ML systems

We know these data types best...

Our professional data crawling experts can help you look into extraction and processing of the following data classes from the public web:

eCommerce Data

Events & Ticketing Data

HR & Recruitment Listings

Education Courses

Ratings & Reviews

Company Profiles

Product & Price Data

News & Media

Travel & Accommodation listings

Take the hassle out of extracting the data you need

Get an active data feed in a matter of days

1. Create enquiry

 

Tell us about the requirements for your project – which type of websites and how often you need the data or leads.

2. Solution architect

 

Our Process Engineer will design a custom scraping plan to fit your requirements

3. Sample Data 

 

We will deliver a sample data report for your report – setting the high quality bar you require, and adjust for any feedback

4. Scale Crawls

 

We will scale up your crawls and deliver accurate and reliable data at a speed you requested

5. Maintenance

 

We will handle any testing or maintenance regarding the data feeds, quality and technical issues that may arise.

We provide customizable web data scraping services to drive your sales, boost your marketing, and supercharge your due diligence.
Our bespoke solutions have helped well-known multinationals with:
 
– Real Time Price Intelligence
 
– HR & Recruitment Intelligence
 
– Lead Generation
 
– Brand Monitoring
 
– Market Research

and many more…
 
Tell us your project requirements and we can get you an accurate quote from our Data Services team.

Frequently Asked Questions

Technically we can scrape and give to you any data you can browse or access on your web brower on a website.

However, there are certain legal considerations that need to be taken into account with every project (no scraping beyond password logins, data privacy, T&Cs) 

We will walk through your data scraping project requirements with you and help guide you along the way on what kind of data is possible to be extracted while following local law and data scraping best practices.

Extracting data from public websites is legal, however, in certain sites have more stringent T&Cs that prohibit or limit data extraction on their sites, and certain countries will have limitations against extracting personal date, have certain data privacy or IP laws, . 

Our team will work with you on your project requirements to help ensure that it follows your local laws and data scraping best practices.

It depends on two factors:
1. How frequently changing the data is. Frequently changing data usually requires a constant data feed subscription.
2. How much filtering/analysis you require of the data extracted. Complex filters and analysis might require more algorithm development.

Our team will understand your project requirements and recommend the best solution for your particular needs.

Once you’ve submitted your initial project requirements, a member of our data systems team will reach out to you to set up a project discovery call. There the data systems planner will discuss your data extraction objectives and what you hope to achieve in detail, and gather the relevant facts needed to help develop the best data-driven solution for your situation. Within a couple of days, we will then present you this planned solution for your approval.

Our systems are built on customised server/browser automation software stacks around the TQ Cloud Scraping Engine of TQ Prospector, available on Google Chrome Store: 

https://chrome.google.com/webstore/detail/tq-prospector-find-emails/cgkdjkpmmhmdpdognlhehmmhndlibfng?hl=en-GB

At ThunderMetrics we develop data extraction systems for businesses that rely on us for timely insights and sales leads. Therefore, our priority is to deliver high quality data that achieves your objectives. Thus, we have a double QA/QC process that monitors the status of the data pipelines and data quality, and our data storage systems run minute by minute rolling backups with < 1 hr RPO, and < 8 hr RTO.

Setup fees depend on the nature and complexity of the project (ad-hoc data scan, data subscription, custom enterprise data analytics), the difficulty and level of complication in the data structure of the websites and quantity of data being extracted. Our data systems team will assess the websites and provide you with a quote for your team / management approval

We offer support to all customers running data pipelines / feed subscriptions as well as ad-hoc single website scans. We will help with data quality issues, exchange of data for invalid data, coverage issues, changes in site structure.When websites have a major structural change, we have a 5-Step Data Scraper Recovery Process in which our teams will redevelop the crawler. We only get paid as long as the data flows.
 

Yes, if sample data is available for the data source of your choice, we are happy to provide it in CSV/JSON format. If it is a new website we have not crawled before – sample data will be provided following crawler development kick off. This occurs post purchasing.

We offer many delivery types such as OneDrive, TQ.CDN, AWS S3, Google Cloud storage, Email, Dropbox and Google Drive. Formats for delivery can be CSV, JSON, or data format of your choice, assuming presence of relevant converters.

Get Sample Data

Fill up the form below and we’ll pass you a link to our sample data for download

Get Sample Data

Fill up the form below and we’ll pass you a link to our sample data for download

Get Sample Data

Fill up the form below and we’ll pass you a link to our sample data for download

Get Sample Data

Fill up the form below and we’ll pass you a link to our sample data for download

Get Sample Data

Fill up the form below and we’ll pass you a link to our sample data for download

Get Sample Data

Fill up the form below and we’ll pass you a link to our sample data for download