setup, config, selenium_driverless, images, comments

[milahu]

• setup: add setup.py, move files to python module
• config: dont read config from config.py
• selenium_driverless: make it work on linux by default
• download images - replace Add option to download images #26
• download comments - fix [Feature request] Comments? #3
  • handle removed comments
  • clickable URLs in comments
• add option to write the original "preloads" data to json files
  • rework or remove JSON_DATA_DIR
• output filepath format: let me set the format of all file paths with string.Template like path/to/${blog_handle}/${post_slug}.md
• repost count ("restack count"). related: like_count_element = soup.select_one("a.post-ufi-button .label")
• post datetime format: should be ISO date format by default, with option to configure date format
• create a main Markdown file readme.md with links to all posts
  • use short date format %Y-%m-%d, not full ISO datetime format
  • list of posts → table of posts?
    • pro: word-wrapping in list makes it hard to see columns (like count...)
    • con: table has cell borders (visual noise)
• convert links to posts in the same publication
• update git repo in output_directory
• add a page footer: Version ${iso_datetime_of_last_change_on_this_page} archived from <a href="${original_post_url}">${original_blog_domain}</a> (this should not use the scrape time to get reproducible output)
• make optional: HTML output
• rework HTML_TEMPLATE and ASSETS_DIR for HTML output
• input faster: email + password
• preserve original post html - related: content_element = soup.select_one("div.available-content")
• remove post footer: ${blog_name} is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Subscribed
• fix off-by-one error in for url in tqdm(self.post_urls, total=total):
• remove NUM_POSTS_TO_SCRAPE
• convert like_count from str to int
• add option to copy assets
• handle timezones? all datetimes are UTC?
• args.number should apply to posts sorted in ascending order, so number=3 selects the 3 oldest posts
• add parameter offline to render Markdown/HTML files from existing JSON files
  • wontfix: post_preloads["post"]["body_html"] != str(soup.select_one("div.available-content")) -- post_preloads["post"]["body_html"] can contain component data that must be rendered to HTML, for example to <div data-component-name="DigestPostEmbed"
  • make the offline parameter work with args.number = preserve JSON data which is out of range