What steps will reproduce the problem? 1. Start rails server ok 2. set account and android id into java code 3. ant building 4. runing java -jar crawler.jar -pname ../../data/pkgreduced pkgreduced is a file with one line: com.whatsapp What is the expected output? What do you see instead? What version of the product are you using? On what operating system? im on debian x64 and using a clone of today's code. Here is the additional information. Bye and good job! Please provide any additional information below. java -jar crawler.jar -pname ../../data/pkgreduced Feb 06, 2012 2:55:47 PM com.marketplace.Main createPackageThread INFO: Creating threads for fetching apps via packages. Feb 06, 2012 2:55:47 PM com.marketplace.io.Fetcher readFile INFO: Read 2 lines Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: API Level: 1 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: com.whatsapp API Level: 1 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: API Level: 2 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: com.whatsapp API Level: 11 Feb 06, 2012 2:55:48 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: com.whatsapp Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: com.whatsapp API Level: 2 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: API Level: 3 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: com.whatsapp API Level: 3 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: API Level: 4 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: com.whatsapp API Level: 4 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: API Level: 5 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: com.whatsapp API Level: 5 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: API Level: 6 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: com.whatsapp API Level: 6 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: API Level: 7 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: com.whatsapp API Level: 7 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: API Level: 8 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: com.whatsapp API Level: 8 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: API Level: 9 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: com.whatsapp API Level: 9 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: API Level: 10 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: com.whatsapp API Level: 10 Feb 06, 2012 2:55:47 PM com.marketplace.service.PackageThread run INFO: Creating thread for package: API Level: 11 Feb 06, 2012 2:55:48 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: Feb 06, 2012 2:55:48 PM com.marketplace.Utils$Permission <init> INFO: Loading permissions Feb 06, 2012 2:55:49 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: com.whatsapp Feb 06, 2012 2:55:49 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: Feb 06, 2012 2:55:50 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: Feb 06, 2012 2:55:50 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: com.whatsapp Feb 06, 2012 2:55:50 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: Feb 06, 2012 2:55:51 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: com.whatsapp Feb 06, 2012 2:55:51 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: Feb 06, 2012 2:55:52 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: com.whatsapp Feb 06, 2012 2:55:52 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: Feb 06, 2012 2:55:52 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: com.whatsapp Feb 06, 2012 2:55:53 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: Feb 06, 2012 2:55:53 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: com.whatsapp Feb 06, 2012 2:55:54 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: Feb 06, 2012 2:55:55 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: com.whatsapp Feb 06, 2012 2:55:55 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: Feb 06, 2012 2:55:56 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: com.whatsapp Feb 06, 2012 2:55:56 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: Feb 06, 2012 2:55:57 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: com.whatsapp Feb 06, 2012 2:55:57 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: Could not initialize class com.marketplace.Utils at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) at java.util.concurrent.FutureTask.get(FutureTask.java:111) at com.marketplace.Main.execute(Main.java:283) at com.marketplace.Main.resolveArgs(Main.java:327) at com.marketplace.Main.main(Main.java:338) Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.marketplace.Utils at com.marketplace.io.Sender$AppQuery.toString(Sender.java:788) at com.marketplace.io.Sender.appExists(Sender.java:117) at com.marketplace.io.Sender.addAppToCollection(Sender.java:282) at com.marketplace.service.PackageThread.run(PackageThread.java:102) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Feb 06, 2012 2:55:57 PM com.marketplace.io.Fetcher getAppsByPackageName INFO: Getting package: com.whatsapp

in crawler/build.xml should change: 1.<fileset dir="/Users/raunak/Development/android-marketplace-crawler/crawler/bin"/> to <fileset dir="bin/"/> 2.add <property name="bin.dir" location="bin" /> and <mkdir dir="${bin.dir}" />

Hi, I tried to launch the crawler but I have some problems with loading permissions. What steps will reproduce the problem? 1. server working properly 2. trying to launch the crawler 3. What is the expected output? What do you see instead? apps I see errors What version of the product are you using? On what operating system? windows vista Please provide any additional information below. this is what I get : C:\android-marketplace-crawler\crawler\dist\lib>java -jar crawler.jar -c oct. 24, 2011 9:37:36 PM com.marketplace.Utils$Permission <init> Infos: Loading permissions Exception in thread "main" java.lang.ExceptionInInitializerError at com.marketplace.util.Device.<init>(Unknown Source) at com.marketplace.util.DeviceInventory.<clinit>(Unknown Source) at com.marketplace.io.SessionManager.initSessions(Unknown Source) at com.marketplace.io.SessionManager.<init>(Unknown Source) at com.marketplace.Main.<init>(Unknown Source) at com.marketplace.Main.main(Unknown Source) Caused by: java.lang.ArrayIndexOutOfBoundsException: 1 at com.marketplace.Utils$Permission.<init>(Unknown Source) at com.marketplace.Utils.<clinit>(Unknown Source) ... 6 more Thanks for help

Ok, server running Crawler running But in the server I get ++ Started POST "/exists.json" for 127.0.0.1 at 2011-10-10 13:46:24 -0400 ActionController::RoutingError (No route matches "/exists.json"): Rendered C:/Ruby192/lib/ruby/gems/1.9.1/gems/actionpack-3.0.3/lib/action_dispatc h/middleware/templates/rescues/routing_error.erb within rescues/layout (2.0ms) Started POST "/exists.json" for 127.0.0.1 at 2011-10-10 13:46:24 -0400 ActionController::RoutingError (No route matches "/exists.json"): Rendered C:/Ruby192/lib/ruby/gems/1.9.1/gems/actionpack-3.0.3/lib/action_dispatc h/middleware/templates/rescues/routing_error.erb within rescues/layout (7.0ms) ++++++++ What does it mean? What should I do? What steps will reproduce the problem? 1. Just running the server 2. Running the crawler 3. What is the expected output? What do you see instead? The apps(?) What version of the product are you using? On what operating system? The last one so far, Windows 7 Please provide any additional information below.

What steps will reproduce the problem? 1. I have problems with any instruction, no one works 2. In this case I introduce java -jar crawler.jar -pname pak.txt 3. I get like answer java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.R untimeException: Response code = 509, msg = unknown at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source) at java.util.concurrent.FutureTask.get(Unknown Source) at com.marketplace.Main.execute(Unknown Source) at com.marketplace.Main.resolveArgs(Unknown Source) at com.marketplace.Main.main(Unknown Source) Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Response code = 509, msg = unknown at com.gc.android.market.api.MarketSession.executeRawHttpQuery(MarketSes sion.java:306) at com.gc.android.market.api.MarketSession.executeProtobuf(MarketSession .java:252) at com.gc.android.market.api.MarketSession.flush(MarketSession.java:221) at com.marketplace.io.Fetcher.getAppsByPackageName(Unknown Source) at com.marketplace.service.PackageThread.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.lang.RuntimeException: Response code = 509, msg = unknown at com.gc.android.market.api.MarketSession.executeRawHttpQuery(MarketSes sion.java:287) ... 10 more Oct 09, 2011 1:20:42 PM com.marketplace.io.Fetcher getAppsByPackageName What is the expected output? What do you see instead? The apps(?) What version of the product are you using? On what operating system? The last one, so far. Windows 7 Please provide any additional information below.

I do not have knowledge about Ruby on Rails, I am having several problems and the first one is to try to set up the server. What steps will reproduce the problem? 1. Install Ruby on Rails 3 2. In C:\Android-Market-Crawler\server> bundle install What is the expected output? What do you see instead? Server working properly(???) What version of the product are you using? On what operating system? Windows 7 Please provide any additional information below. Fetching git://github.com/adamcooper/vestal_versions C:/Ruby192/lib/ruby/gems/1.9.1/gems/bundler-1.0.21/lib/bundler/source.rb:579:in ``': No such file or directory - git clone "git://github.com/adamcooper/vestal_v ersions" "C:/Ruby192/lib/ruby/gems/1.9.1/cache/bundler/git/vestal_versions-521a0 96e20ac6a423069756d4876cfa0e693286d" --bare --no-hardlinks (Errno::ENOENT) from C:/Ruby192/lib/ruby/gems/1.9.1/gems/bundler-1.0.21/lib/bundler/sour ce.rb:579:in `git'

When running crawler.jar, an error response says "A CAPTCHA is required. (A response with this error code will also contain an image URL and a CAPTCHA token.) What steps will reproduce the problem? 1. build using ant 2. run java -jar crawler.jar birds This was done using Ubuntu 10.10, Ruby 1.9. Also tested on another ubuntu based box.

The fileset directory has a path of /User/authorsname/development/etc This causes a build error unless it's manually set