Thursday, November 2, 2017

Quick Update - In Other News

I had to dedicate a post specifically for the EvoWars beta release!  And I'm very excited about that.

In other news...

Financial Scrape Project
My large scrape project with buddies have fallen off track... everyone has been busy and that hasn't progressed.  We have day jobs, so it's to be expected...

Daddy's Diaper Deals
The scrape bot that I was putting together with an offshore resource did well initially, but Amazon scraping is a beast.  They have rotating layouts and overlays that really mess things up sometimes... We had a prototype python code using scrapy, but it wasn't robust enough to do what I wanted.

I'm going to revisit this project during the month of November.  That's going to the priority project (even if that means putting off EvoWars until Dec).  I'm still planning on using Scrapy, but I'm going to try some other features like deploying in their cloud environment.

If you want to learn scraping, check Scrapy out and try out Portia to learn the patterns and syntax.  Really simple framework and great tutorials on their Scraping Hub site.

--
That's is it for today.  Just wanted to provide a brief update on what is upcoming this month.

Cheers!
Alex

No comments:

Post a Comment