https://blog.kenhuang.io/tags/scrapy/ Recent content in Scrapy on Ken Huang Hugo en whatacold@gmail.com (Ken Huang) whatacold@gmail.com (Ken Huang) Wed, 26 Feb 2025 00:33:21 +0800 https://blog.kenhuang.io/blog/2022-04-09-scrapy-csv-without-header/ Sat, 09 Apr 2022 22:32:11 +0800whatacold@gmail.com (Ken Huang) https://blog.kenhuang.io/blog/2022-04-09-scrapy-csv-without-header/ <figure> <img src="https://docs.scrapy.org/en/latest/_images/scrapy_architecture_02.jpg" alt="Scrapy Architecture" title="Click to enlarge the image"/> <figcaption> Scrapy Architecture </figcaption> </figure> <p> Scrapy provides a few <a href="https://docs.scrapy.org/en/latest/topics/exporters.html">item exporters</a> by default to export items in commonly used file formats like CSV/JSON/XML. I usually use CSV to export items, it is pretty convenient, and it comes in two ways:</p> <ul> <li>appending mode, for example, <code>scrapy crawl foo -o test.csv</code></li> <li>overwriting mode with <code class="verbatim">-O</code> option, like <code>scrapy crawl foo -O test.csv</code></li> </ul> <p>But in the appending mode, it&#39;s a bit annoying that it always appends the header row before the newly scraped items, which is not correctly in terms of CSV format.</p>