Not sure what happened to the most recent job description, so updating it below. Please review to ensure you can satisfy all requirements.
Seeking data mining automation professional to write, test, and implement the following by July 1, 2018.
– Logs in to authenticated website #1, which is the initial source of data. This data will be used to create new rows in Google Sheets. All subsequent data collected in this process will be added as new data in additional columns to the rows of data from website #1.
– Choose values in a couple of different dropdown lists
– Update filter values in text boxes
– Automate button or link clicks
– web scrape and clicking on links and icons to download CSVs
– Take CSV, then map data columns and append into existing Google Sheets
– Log into another authenticated website #2
– Download CSVs from site #2, perform various aggregate calculations on data from CSV, then add calculated data into new columns to above appended rows in Google Sheets
– Return to authenticated website #1, and choose different dropdown and filter values
– Rinse & repeat process from there
– Provide a console interface to allow us to modify values for dropdown & text filters, and increments.
– Incorporate randomized but short pauses during steps.
Project requirement also involves assisting deploying this solution onto a brand new cloud VM. Right now, vultr.com is our preferred provider.
We would like to run a reasonable number of concurrent processes above either on the same VM or 1 process to 1 VM.