Knowledge From Data

The Blog of Datafiniti Inc.

Austin A-List & Upcoming Improvements

leave a comment »

So this is exciting.  Within 6 months of moving our team to Austin, we’ve been named by the Austin Chamber of Commerce as one of their 2013 A-List companies.  We were included in the “Emerging” category and selected from a group of 157 companies.  You can read more here: http://impactnews.com/austin-metro/central-austin/%27a-list%27-companies-named/.

It’s always nice to get recognition, but we still have a lot of work to do to achieve our goal of making web data fully accessible.  Earlier this week we gave early access to our V2 API through a new Download App.  The new version of our website and API will be made publicly available next week.  We’re still considering ourselves to be in beta mode, though, so please give us any feedback you have.

In addition to these upcoming releases, we’re also working on a giant upgrade to our back-end architecture to dramatically improve the volume and rate of web content we’re ingesting.  We have a metrics we’re targeting:

  • Crawling individual, content-rich websites at a rate of 100,000 URLs per day.  Each daily website-specific daily crawl will track high-priority businesses and products to provide daily-updated data on reviews and prices.
  • Crawling more than 1 billion URLs every month.  This will enhance our web-wide crawling for more content/data discovery on businesses, people, and products.

Meeting these goals is our focus for the next 2 months, so that starting from January, we’ll be in a great position to provide significantly more up-to-date data to our customers.

Written by datafiniti

November 8, 2013 at 12:45 pm

Posted in Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s