Saturday, 26 March 2011

IMACRO & Application Development for Data Scraping...

  • Posted: Thu, Mar 24, 2011                    
  • Budget: Less than $500
  • Fixed Price Job
  • Guaranteed with Elance Escrow
  • W9 Not Required    
  • Time Left: 5d, 3h
  • Location: Anywhere
  • Start: Immediately
Developer’s skill set needed:

- Understand iMacros (software and use) and proficient development with iMacros commands and variables (iOpus products) if we are going with Option #1 (see software manual here)
- Scripting Language – C++ programming/JavaScript/VBscript/Perl – whichever is more commonly used.
- Web Applications/Development
- .Net Development
- Web and Data Scraping Procedures (data processing)
- Software Architecture/XML/PHP/HTML/AJAX/JAVA/C# - Backend and GUI development expertise needed.

Work description:

We are looking for an experienced programmer/developer with advanced expertise in iMacros, web and application development. Currently, we have a set of manual procedures that our VA runs through once a week gathering foreclosure publication data, and entering the data into an excel spreadsheet on a weekly basis. We are interesting in automating these procedures by using the iMacros tool to develop templates to mimic the human interactions with various county websites that we work with in our manual procedures. We need a master scripting guru who is proficient with using the iMacros software to be able to write action scripts and customize the iMacros commands to perform and automate each step in our manual procedures.

The websites we deal with are a bit complex in terms of structure – so advance scripting knowledge is required (i.e. we need to be able to automate the task of running a query on a specific website, propagating through a list of the search results, capturing specific text in the foreclosure publication and looping for every record in the search result). The script/macros will need to be written dynamically to support any data or website content changes. There is a set of about 12 procedures with approximately 6 having unique procedures.

In addition, iMacros is NOT able to perform the same extraction/text recognition recording on a pdf file. A few of the websites in our procedures offer the foreclosure data publication via a pdf attachment. We need a developer to come up with a design and a solution to develop this functionality and integrate the solution with iMacros’ scripts.

We already have an existing text-scrapping program that the developer can use as a foundation to build from. How the program currently works is that it requires manually pasting a publication text before parsing the data into excel with the correct format. This task will need to be automated. Ultimately. the goal is to be able to run a single program to initiate the macro/scripts and capture/process data on the web or text files with little or no interaction by the User, automatically. If the software can automate the existing manual procedures, export/parse the data and perform calculations, send the final data to defined users, and do so without any errors, then it meets project requirements. Example of end data results with notes/comments and additional details will be submitted to the selected bidder.

At a high-level, this project contains 4 major tasks:

1. Customize scripts/macros to automate the manual tasks in our procedure using the iMacros platform.
2. Develop a plug-in or a program to scrap text offline and from any text file – pdf, html, .doc, not just websites. Then integrate this solution with iMacros scripts or vice versa.
3. Develop an excel plug-in or script to parse data into the format required, perform data calculation, categorize list into new worksheet based on attributes.
4. Schedule an automatic delivery of parsed/final data to recipients per county or a master list with data from all counties merged.

H-L Functional Requirements will be provided to selected bidder.

No comments: