# Tor Browser sequential scraper — proof of work **Job:** `caf4bee4-c708-499c-9e9c-27ecb0a1408d` — Python Developer: Tor Browser Automation Script (Manual CAPTCHA Pause + CSV Output) **Status:** built and tested before applying. ## What was tested - Windows 11, Python 3.10, Selenium 4.52.0, geckodriver 0.37.1, Tor Browser 16.0. - 13 unit tests passed. - End-to-end integration test passed against a local HTTP fixture in headless Tor Browser: the script opened both pages sequentially and produced the exact CSV below. - The same end-to-end test passed with the real proxy path: the script started Tor Browser's bundled `tor.exe`, waited for SOCKS `127.0.0.1:9150`, launched the actual Tor Browser binary with a copy of the actual Tor Browser profile, and loaded the local fixture through the proxy-bypass rule. ## Actual test output ```csv City,Quantity,Price,Store Alpha,2,$10.00,Alpha Store Beta,1,€20.50,Beta Store ``` ## Interface ```bash python tor_scraper.py \ --links links.txt \ --selectors selectors.json \ --output output.csv \ --tor-browser "C:\Users\me\Desktop\Tor Browser" ``` `links.txt`: ```csv City,Quantity,URL London,2,https://example.com/product/1001 Berlin,1,https://example.com/product/1002 ``` `selectors.json`: ```json { "price": ["css:.product-price", "xpath://span[@itemprop='price']"], "store": ["css:.store-name", "xpath://h1[contains(@class,'store')]"], "captcha": ["css:iframe[src*='captcha']"] } ``` The client's real selectors and links can be dropped in without code changes. ## Key behavior The first page pauses for manual verification; later pages only pause when a CAPTCHA is detected or a page/value fails: ```python if first and args.pause_first: action = wait_for_enter( "First page is open. Solve any CAPTCHA or verification in the Tor Browser window.\n" "Do not press Enter until the page you want to scrape is fully visible.", args.no_pause, ) _raise_on_quit(action) if not (first and args.pause_first) and detect_captcha(driver, selectors.get("captcha", [])): action = wait_for_enter( "A CAPTCHA or verification page is showing. Solve it in the Tor Browser window, then continue.", args.no_pause, ) ``` Same tab, same profile, same session: ```python driver.get(row.url) # one tab, sequential price = extract_value(driver, selectors["price"], args.selector_timeout) store = extract_value(driver, selectors["store"], args.selector_timeout) ``` Exact CSV header and incremental flush: ```python writer = csv.writer(output_file, lineterminator="\n") writer.writerow(["City", "Quantity", "Price", "Store"]) ... writer.writerow([row.city, row.quantity, price, store]) output_file.flush() ``` ## Security / static analysis - No `requests`, `urllib`, `http.client`, or other outbound HTTP libraries. - No `eval`, `exec`, `compile`, `base64`, `pickle`, or `marshal`. - No obfuscation or encoded payloads. - No telemetry or third-party endpoints. - The only network navigation is Selenium opening the URLs in the client's input file. - `subprocess` is used only to launch Tor Browser's bundled `tor.exe` with an argument list; `shell=True` is never used. The exact command is logged at startup. - The temporary browser profile is deleted on exit unless `--keep-profile` is passed. - The script never solves or bypasses CAPTCHAs. ## Delivery On acceptance I will deliver: - `tor_scraper.py` — full source. - `README.md` — install, selectors, run, troubleshooting. - `requirements.txt`. - `links.sample.txt` and `output.sample.csv`. - Unit and integration tests, including the local fixture pages. - One round of selector adjustments against the client's real pages. Fixed price: **1.5 XMR**. Estimated duration: immediate delivery; 24 hours for selector adjustments. The profile wallet is already configured to receive XMR.