Source of data

**jeep** Fri Mar 18, 2005 6:51 pm

Source of data
Blast,

I wonder if there's any source to find collected financial results (such as quarterly earnings during the past decade) of the companies. Yahoo has only three years of annual data or three quarters of quarterly data.

I know you can dig into the 10K's to find those numbers at SEC. Is there any conveneint place to find such data? If not, how do you suggest if one wants to crawl such data using a program?

Thanks.

**blast_investor** Fri Mar 18, 2005 7:54 pm

I am not aware of datasource for historical data other than SEC filings.

I can search google to see if there is any place to offer this historical data for free or small fee.

**goog** Mon Mar 21, 2005 11:45 am

morningstar.com has ten years data

**jeep** Mon Mar 21, 2005 3:31 pm

goog: this makes it handy. thanks.

Another question: blast, where do you pull off the stock ticker list? Yahoo has it, but you have to get into it alphabetically and not convenient for downloading purpose.

**blast_investor** Mon Mar 21, 2005 11:36 pm

BlastInvest runs a web robot over web grapping stock ID from Yahoo automatically.

True, it is not convenient downloading IDs manually. But I do not have to do it manually any way.

**jeep** Tue Mar 22, 2005 2:37 pm

Blast,

May I ask which programming language do you use? Python (this is more modern than Perl)? Perl? NQL? Do you use statistical analysis packages like R after pulling off the data?

Thanks.

blast_investor wrote: BlastInvest runs a web robot over web grapping stock ID from Yahoo automatically.

True, it is not convenient downloading IDs manually. But I do not have to do it manually any way.

**afan** Tue Mar 22, 2005 7:15 pm

co-ask

and I have some more questions. As Blast said, a robot to dig data from Yahoo is a quite important component of screen tool in choosing stocks. How can we newbies learn the techniques to develop a robort for ourselves? Can Blast suggest some basic books or websites?

Thanks a lot:)

jeep wrote: Blast,

May I ask which programming language do you use? Python (this is more modern than Perl)? Perl? NQL? Do you use statistical analysis packages like R after pulling off the data?

Thanks.

blast_investor wrote: BlastInvest runs a web robot over web grapping stock ID from Yahoo automatically.

True, it is not convenient downloading IDs manually. But I do not have to do it manually any way.

**blast_investor** Tue Mar 22, 2005 8:27 pm

BlastInvest does not disclose our software tool technology details.

So sorry, I can not provide more technology details in this part.

**blast_investor** Tue Mar 22, 2005 8:40 pm

You can search Google for terms:

"web robot" or "web programming" etc. to get more information.

afan wrote: co-ask

and I have some more questions. As Blast said, a robot to dig data from Yahoo is a quite important component of screen tool in choosing stocks. How can we newbies learn the techniques to develop a robort for ourselves? Can Blast suggest some basic books or websites?

Thanks a lot:)

**afan** Tue Mar 22, 2005 10:27 pm

3x a lot. It is quite understandable that you keep the secret. Your hint is quite helpful, I should have been searching them before:)

BTW, since I have been familar to my computer language, and have used it to fetch insider buying information, I don't care which language you are using. But I don't want to interupt jeep's question about the languge, so if you like, you can tell him the answer of his question. :oops:

blast_investor wrote: You can search Google for terms:

"web robot" or "web programming" etc. to get more information.

afan wrote: co-ask

and I have some more questions. As Blast said, a robot to dig data from Yahoo is a quite important component of screen tool in choosing stocks. How can we newbies learn the techniques to develop a robort for ourselves? Can Blast suggest some basic books or websites?

Thanks a lot:)

**blast_investor** Tue Mar 22, 2005 11:31 pm

You are welcome.

Yeah, the non-technology disclosure policy is for competitive reasons.

afan wrote: 3x a lot. It is quite understandable that you keep the secret. Your hint is quite helpful, I should have been searching them before:)

**jeep** Wed Mar 23, 2005 12:34 am

I just want to pick an efficient language. Well, in any case I probably will use Python anyway. Can't believe there are 30,000+ tickers at yahoo. Insider info is more trouble if you want to scan SEC directly since they don't use stock tickers but CIK codes...

I respect blast's choice to keep it but I do think the real trade secret here is the algorithm to find potentials. Other technical stuff, like crawling, can be easily achieved, just a matter time.

afan wrote: 3x a lot. It is quite understandable that you keep the secret. Your hint is quite helpful, I should have been searching them before:)

BTW, since I have been familar to my computer language, and have used it to fetch insider buying information, I don't care which language you are using. But I don't want to interupt jeep's question about the languge, so if you like, you can tell him the answer of his question. :oops:

blast_investor wrote: You can search Google for terms:

"web robot" or "web programming" etc. to get more information.

afan wrote: co-ask

and I have some more questions. As Blast said, a robot to dig data from Yahoo is a quite important component of screen tool in choosing stocks. How can we newbies learn the techniques to develop a robort for ourselves? Can Blast suggest some basic books or websites?

Thanks a lot:)

**ffreedom** Wed Mar 23, 2005 1:26 am

This book is all you need to scrape Yahoo.
Fetching data from Yahoo is quite easy. Just read this book and you will be a master of that.

http://www.oreilly.com/catalog/spiderhks/

But Blast already does it for you in his free Insider Buy Yahoo group. Why do you bother to do it? The true strength of Blast is to sift through those insider buys and find the real gems. That's the hardest part in my opinion ...

afan wrote: co-ask

and I have some more questions. As Blast said, a robot to dig data from Yahoo is a quite important component of screen tool in choosing stocks. How can we newbies learn the techniques to develop a robort for ourselves? Can Blast suggest some basic books or websites?

Thanks a lot:)

jeep wrote: Blast,

May I ask which programming language do you use? Python (this is more modern than Perl)? Perl? NQL? Do you use statistical analysis packages like R after pulling off the data?

Thanks.

blast_investor wrote: BlastInvest runs a web robot over web grapping stock ID from Yahoo automatically.

True, it is not convenient downloading IDs manually. But I do not have to do it manually any way.

**vin** Wed Mar 23, 2005 10:22 am

agree.

develop parsing tools is easy. and there are insider transcation pages all over the web. I developed the real time insider parsing program, but that will not offer a lot of advantage.

**memento** Wed Jun 15, 2005 9:38 am

Re: This book is all you need to scrape Yahoo.
chihwei_chang wrote: But Blast already does it for you in his free Insider Buy Yahoo group.

I am a newbie here :oops: :oops: could you tell me where this group is?

Thanks a lot