__/ [ Satchel ] on Friday 17 March 2006 23:39 \__
> Roy Schestowitz wrote:
>
>> __/ [ Larry ] on Thursday 16 March 2006 13:18 \__
>>
>>> I have just started using Debian but have been around computing for 20
>>> years.
>>>
>>> What I want to do is capture the market data my broker feeds to me via
>>> their trading page. the data is not streaming, but changes fairly often.
>>> The web page containing the data is constantly updating,
>>> except I want to be able to take a snapshot about 20 times per hour for
>>> the "put/call" prices at a specified time so I can have a historical
>>> timeline. The broker does not provide a historical/daily/minute updates
>>> in a file format I can use so it is up to me to capture this data and
>>> massage it into a format I can use.
>>
>> Scrape it, then interpret and process it.
>>
>> ,----[ Command ]
>> | man wget
>> `----
>>
>> If the data is provided in the form of feeds, you might also wish to parse
>> it using tools from the RSS 'family', e.g.:
>>
>> http://magpierss.sourceforge.net/
>>
>>
>>> Several years ago I wrote some "c" file utilities that would parse out
>>> bad data from downloads but I don't think the above described application
>>> would be the same as this was kinda pre internet....still using Bulletin
>>> Boards back then with a 300 baud modem. Now, we have html,xml, etc and I
>>> was wondering how to get started capturing this information.
>>>
>>> I wish I could find a macro, as you see I don't know what is available
>>> but I have googled to death.
>>
>>
>> Let me know it you need further help because I have some useful scripts at
>> hand.
>>
>>
>>> Thanks in advance for any suggestions.
>>>
>>> Larry
>>> Birmingham, Al.
>>
>>
>> Hope it helps,
>>
>> Roy
>>
> Thanks Roy,...I will be calling upon your advise
__/ [ Satchel ] added on Friday 24 March 2006 14:52 \__
> Ok...I was away for a while. You said "Scrape it". I suppose that is a
> script of somekind. I haven't a clue what you meant
Hi Satchel,
"Scraping" is just a term, which often refer to the act of taking content
from one site and using it in another. It *might* also apply to the scanrio
where content is fetched in unexpected ways (e.g. spidering), as in this
case. Depending on the process/intent for which you fetch the data, I guess
the terminology (affected by and dependent on ethics) will vary. Scraping
has a very gross connotation.
Best wishes,
Roy
--
Roy S. Schestowitz | while (sig==sig) sig=!sig;
http://Schestowitz.com | SuSE Linux ¦ PGP-Key: 0x74572E8E
3:40pm up 16 days 8:17, 8 users, load average: 0.98, 0.82, 0.81
http://iuron.com - next generation of search paradigms
|
|
