The blog post goes into the specifics of the incident, along with some changes that Open3DLab has made since to try and avoid a repeat. According to the post, Open3DLab's staff noticed a crawler "making a large number of requests to our sites and download infrastructure" on October 18. The crawler used trellis2-data-sourcing/1.0 (3D foundation model research; [redacted]@roblox.com) as its user agent, with the data coming from a single Amazon AWS address. The crawler didn't respect the site's robots.txt file or Terms of Service, which explicitly forbid scraping. It also, according to the report, ignored creator licenses for the projects, which prohibit non-commercial use.
The crawler read "practically [its] entire catalogue" of 18,926 projects in a span of 34 hours over a total of 26,429 requests. Once it was done, the crawler downloaded 1,027 files from 712 projects before it "suspended operations." Files downloaded include 364 files, or 54.8GB worth, from 224 projects hosted on SmutBase, OpenLab's sister site focused on adult content. A further 345 projects of adult-marked SFMLab projects, or 62.6GB worth, were also taken.
According to Open3DLab, the crawler's "main limiting factor" was its use of free download servers that capped speed and concurrent downloads. The scraping was detected on the same day, and blocked by the following morning. Further download attempts were detected on October 19, "neatly within office hours, only briefly pausing for an hour around lunchtime," and despite being blocked, "did not reduce its download attempts."
A final attempt on October 20, with requests tapering, "likely in an effort to test… rate limiting," was made, with requests stopping completely later that day. Open3DLab made another request at this point for comment, although without reply.
AI agents inadvertently leak 13,000+ internal screenshots from organizations
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/tech-industry/artificial-intelligence/SPONSORED_LINK_URL
- https://www.tomshardware.com/tech-industry/artificial-intelligence/open3dlab-accuses-roblox-of-downloading-145gb-of-3d-assets-for-ai-training-purposes-automated-crawler-ignored-site-rules-to-grab-over-1-000-files-including-adult-content-during-scraping-spree#main
- https://www.tomshardware.com/membership
- Save $134 on this WD Red Plus 10TB HDD bundle at Newegg
- Nvidia RTX 20, 30, and 40-series GPUs hacked to remove a half-speed AI throttle
- Leading European Financial Services Organizations Embrace Open Models to Build Proprietary Intelligence
- Chinese research vessels loiter near Pacific undersea cables more than any other nation, report finds
- Intel quietly confirms Raptor Lake Next CPUs through internal driver publication
Informational only. No financial advice. Do your own research.