The blog post goes into the specifics of the incident, along with some changes that Open3DLab has made since to try and avoid a repeat. According to the post, Open3DLab's staff noticed a crawler "making a large number of requests to our sites and download infrastructure" on October 18. The crawler used trellis2-data-sourcing/1.0 (3D foundation model research; [redacted]@roblox.com) as its user agent, with the data coming from a single Amazon AWS address. The crawler didn't respect the site's robots.txt file or Terms of Service, which explicitly forbid scraping. It also, according to the report, ignored creator licenses for the projects, which prohibit non-commercial use.
The crawler read "practically [its] entire catalogue" of 18,926 projects in a span of 34 hours over a total of 26,429 requests. Once it was done, the crawler downloaded 1,027 files from 712 projects before it "suspended operations." Files downloaded include 364 files, or 54.8GB worth, from 224 projects hosted on SmutBase, OpenLab's sister site focused on adult content. A further 345 projects of adult-marked SFMLab projects, or 62.6GB worth, were also taken.
According to Open3DLab, the crawler's "main limiting factor" was its use of free download servers that capped speed and concurrent downloads. The scraping was detected on the same day, and blocked by the following morning. Further download attempts were detected on October 19, "neatly within office hours, only briefly pausing for an hour around lunchtime," and despite being blocked, "did not reduce its download attempts."
A final attempt on October 20, with requests tapering, "likely in an effort to test… rate limiting," was made, with requests stopping completely later that day. Open3DLab made another request at this point for comment, although without reply.
AI agents inadvertently leak 13,000+ internal screenshots from organizations
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/tech-industry/artificial-intelligence/SPONSORED_LINK_URL
- https://www.tomshardware.com/tech-industry/artificial-intelligence/open3dlab-accuses-roblox-of-downloading-145gb-of-3d-assets-for-ai-training-purposes-automated-crawler-ignored-site-rules-to-grab-over-1-000-files-including-adult-content-during-scraping-spree#main
- https://www.tomshardware.com/membership
- Keybytes ‘truly modular keyboard’ starts with one key and is assembled like Legos
- NVIDIA Open Models Power a New Wave of European Public Sector Projects
- Israeli ad tech firm tied to suspected Starlink tracking effort, report claims
- Save up to $800 on Acer’s ultra-slim RTX 5070 Ti gaming laptop — pay $2225 for the 16GB RAM model or upgrade to 32GB for $2400
- TSMC reportedly weighs 20% CoWoS price increase as Nvidia Blackwell and Rubin chips put pressure on packaging capacity
Informational only. No financial advice. Do your own research.