Visual Web Ripper Logo Visual Web Ripper Logo
Welcome Guest Search | Active Topics | Log In | Register

Tag as favorite
multiple tables in output
digiloop
#1 Posted : Thursday, November 04, 2010 3:15:52 PM
Groups: Registered
Joined: 10/26/2010
Posts: 34
Hi
I have created my project, and there is one problem. project output multiple tables to mysql, even my purpose is to get only last template captured data as results.
I quite deep browsing tree to actual page what contains data, and i have created discovery to that page using list option, and inside lilst there is sub-list (4 level)
Now it seems that every sub-list is creating own table to mysql, and push those results also to db. It is really easy to drop tables, but i know there is some way to void that.
Can you tell where toread or is there easy way to solve that.
Sequentum Support
#2 Posted : Friday, November 05, 2010 1:14:16 AM

Groups: Administrators
Joined: 4/10/2010
Posts: 1,239
Location: Sydney, Australia
digiloop
#3 Posted : Friday, November 05, 2010 3:54:35 AM
Groups: Registered
Joined: 10/26/2010
Posts: 34
Thank, that solve issue. 1 thing is still making me confused. I try to put data row cahce value, but it seems that it is not stored. So is there own Data row cahce value for each individual template. If so is there place to put some default value somewhere instead 0.

Also i want to only have new entry's in DB, to void duplicates. Is best way to do that in database rules, or is there way to do that in ripper tool.

-Pekka
Sequentum Support
#4 Posted : Friday, November 05, 2010 6:38:03 AM

Groups: Administrators
Joined: 4/10/2010
Posts: 1,239
Location: Sydney, Australia
You should not set the "Data row cache" option on more than one template. See this article for more information:

http://manual.visualwebr...ault.aspx?manual_id=675

Please read the following article for more information about incremental web scraping:

http://manual.visualwebr...fault.aspx?manual_id=344
digiloop
#5 Posted : Thursday, November 11, 2010 1:48:14 PM
Groups: Registered
Joined: 10/26/2010
Posts: 34
Hi

i have trying to put values between 1 to 500000 and still seems that almost every page will create input to DB, i use mysql. Also i find every template in project and change advanced options data row cache value from 0 to same what i put /project/project options/ tab where is also advanced tab.
Is row idication result row's or is it something else, i am confused. I try this with apple.rip demo project and purpose is to get project put only every 1000 row so DB to increase speed.
Sequentum Support
#6 Posted : Thursday, November 11, 2010 3:59:48 PM

Groups: Administrators
Joined: 4/10/2010
Posts: 1,239
Location: Sydney, Australia
Please attach the project here.
digiloop
#7 Posted : Friday, November 12, 2010 7:27:58 AM
Groups: Registered
Joined: 10/26/2010
Posts: 34
Here it is.

File Attachment(s):
Apple_fi.rip (107kb) downloaded 28 time(s).
Users browsing this topic
Guest
Tag as favorite
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

Powered by YAF 1.9.4 RC1 | YAF © 2003-2009, Yet Another Forum.NET
This page was generated in 0.089 seconds.