mirror of
git://github.com/kovidgoyal/calibre.git
synced 2025-12-17 22:54:29 +01:00
Compare commits
No commits in common. "master" and "v5.33.0" have entirely different histories.
1168 changed files with 127314 additions and 160823 deletions
2
.github/workflows/ci.yml
vendored
2
.github/workflows/ci.yml
vendored
|
|
@ -7,7 +7,7 @@ jobs:
|
|||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
matrix:
|
||||
os: [ubuntu-latest, macos-latest, windows-2019]
|
||||
os: [ubuntu-latest, macos-latest, windows-latest]
|
||||
steps:
|
||||
- name: Checkout source code
|
||||
uses: actions/checkout@master
|
||||
|
|
|
|||
3
.github/workflows/translations.yml
vendored
3
.github/workflows/translations.yml
vendored
|
|
@ -27,6 +27,9 @@ jobs:
|
|||
- name: Install calibre dependencies
|
||||
run: setup/arch-ci.sh
|
||||
|
||||
- name: Install translations dependencies
|
||||
run: python -m pip install transifex-client
|
||||
|
||||
- name: Bootstrap calibre
|
||||
run: runuser -u ci -- python setup.py bootstrap --ephemeral
|
||||
|
||||
|
|
|
|||
2
.gitignore
vendored
2
.gitignore
vendored
|
|
@ -10,7 +10,6 @@ compile_commands.json
|
|||
link_commands.json
|
||||
src/calibre/plugins
|
||||
resources/images.qrc
|
||||
resources/icons.rcc
|
||||
manual/generated
|
||||
manual/locale
|
||||
manual/.doctrees
|
||||
|
|
@ -59,4 +58,3 @@ recipes/debug
|
|||
/*env*/
|
||||
cmake-build-*
|
||||
bypy/b
|
||||
bypy/virtual-machines.conf
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
Files: *
|
||||
Copyright: Copyright (C) 2008-2022 Kovid Goyal <kovid@kovidgoyal.net>
|
||||
Copyright: Copyright (C) 2008-2020 Kovid Goyal <kovid@kovidgoyal.net>
|
||||
License: GPL-3
|
||||
The full text of the GPL is distributed as in
|
||||
/usr/share/common-licenses/GPL-3 on Debian systems.
|
||||
|
|
|
|||
236
Changelog.txt
236
Changelog.txt
|
|
@ -6,7 +6,7 @@
|
|||
# to the ticket list.
|
||||
# Also, each release can have new and improved recipes.
|
||||
|
||||
# {{{ 5.x.0 2022-xx-xx
|
||||
# {{{ 5.x.0 2021-xx-xx
|
||||
#
|
||||
# :: new features
|
||||
#
|
||||
|
|
@ -23,234 +23,7 @@
|
|||
# - title by author
|
||||
# }}}
|
||||
|
||||
{{{ 5.38.0 2022-03-04
|
||||
|
||||
:: new features
|
||||
|
||||
- [1852929] E-book viewer: When displaying estimated time to completion for reading a book, remember the reading rate the next time the book is opened
|
||||
|
||||
- [1961500] Dark theme: Highlight the current cell in the book list with a lighter background and different foreground to make it more obvious
|
||||
|
||||
- [1961639] An option to disable editing composite columns in the main book list when Tabbing through them (Preferences->Look & feel->Edit metadata)
|
||||
|
||||
:: bug fixes
|
||||
|
||||
- Tag editor: Fix regression in previous release that caused double clicking on tags to not work on non Linux platforms
|
||||
|
||||
- [1962365] Copy to library: Fix annotations not being copied
|
||||
|
||||
- [1962213] Edit book: Spell check: Fix words after a comment not being checked
|
||||
|
||||
- [1960554] PDF Output: Fix conversion failing if there are ToC entries pointing to removed content
|
||||
|
||||
- [1961775] E-book viewer: Fix an error when opening books with MathML for the second time if the last read position was at a MathML element
|
||||
|
||||
- Edit book: Fix double clicking to select a word also selecting smart quotes surrounding the word
|
||||
|
||||
- EPUB 3 metadata: Fix non-integer series index being sometimes represented using exponential notation
|
||||
|
||||
:: improved recipes
|
||||
- Lenta.ru and aif.ru
|
||||
- Indian Express
|
||||
- Live Mint
|
||||
- Mainichi
|
||||
- Japan Times
|
||||
|
||||
:: new recipes
|
||||
- Hindustan Times by unkn0wn
|
||||
- India Legal Magazine by unkn0wn
|
||||
- RT на русском by Vuizur
|
||||
}}}
|
||||
|
||||
{{{ 5.37.0 2022-02-18
|
||||
|
||||
:: new features
|
||||
|
||||
- [1961129] Book details: Add actions to trim the cover to the right-click menu
|
||||
|
||||
- [1960586] Allow removing multiple email addresses at once in Preferences->Sharing by email
|
||||
|
||||
- Book details: Use a better mono-spaced font on Windows by default
|
||||
|
||||
- Add a tweak in Preferences->Tweaks to change the behavior of the Tab key when completing entries
|
||||
|
||||
- [1959928] Edit metadata: In "All in one" mode add an adjustable splitter between the cover and formats boxes
|
||||
|
||||
:: bug fixes
|
||||
|
||||
- [1960686] Textile output: Dont fail if input document has invalid padding or margin specifications
|
||||
|
||||
- [1960446] E-book viewer: Fix image display window not remembering its size and settings when run from within calibre
|
||||
|
||||
- E-book viewer: Fix setting to use roman numerals for series not being respected
|
||||
|
||||
- Edit book: When saving a copy do not fail if the original file has no write permissions
|
||||
|
||||
- [1960180] Embed fonts tool: Create <head> when missing
|
||||
|
||||
- Tag editor: Improve performance when very large number of tags present
|
||||
|
||||
:: improved recipes
|
||||
- Live Mint
|
||||
- The Hindu
|
||||
- Reuters
|
||||
- MMC RTV Slovenija
|
||||
- Down To Earth
|
||||
- Publico.PT
|
||||
}}}
|
||||
|
||||
{{{ 5.36.0 2022-02-04
|
||||
|
||||
:: new features
|
||||
|
||||
- Edit metadata dialog: Allow controlling which custom columns are present in this dialog via Preferences->Look & feel->Edit metadata
|
||||
|
||||
- Edit metadata dialog: Allow manually sizing the various sections of the dialog in "All on 1 tab" mode
|
||||
|
||||
- Edit book: Spell checking: Update the bundled English and Spanish dictionaries
|
||||
|
||||
- [1958773] BibTeX catalogs: Support tags like custom columns
|
||||
|
||||
:: bug fixes
|
||||
|
||||
- [1959659] Amazon metadata download: Fix paragraphs in the comments being merged
|
||||
|
||||
- [1958979] Amazon.de metadata download: Fix published date and series information not being fetched for some books
|
||||
|
||||
- Email delivery: Fix sending email via Hotmail not working since this week because Microsoft changed the SMTP server name
|
||||
|
||||
- [1959220] Do not remove articles for titles in the Polish language
|
||||
|
||||
- [1959207] E-book viewer: When using Read aloud do not automatically lookup the highlighted word until read aloud is paused or stopped
|
||||
|
||||
- E-book viewer: Fix Ctrl+p shortcut for printing not working
|
||||
|
||||
- [1958882] Show an error when viewing a specific format and the file is missing
|
||||
|
||||
- Edit book: Fix renaming of classes that start/end with non word characters not working
|
||||
|
||||
- [1958730] Edit book: Preview panel: Fix hyphenation at end of line being rendered as boxes on macOS
|
||||
|
||||
- [1959893] Fix incorrect selection size displayed in Trim image dialog when image is scaled down to fit
|
||||
|
||||
- [1959782] Edit book: Fix pasting files from another editor instance failing if a file with the same name already exists
|
||||
|
||||
- [1959981] When reviewing metadata if the newly downloaded metadata has no language but there is an existing language, ensure it is preserved
|
||||
|
||||
:: improved recipes
|
||||
- India Today
|
||||
- Indian Express
|
||||
- Live Mint
|
||||
- Al Jazeera in English
|
||||
- The Financial Express
|
||||
- The Straits Times
|
||||
|
||||
:: new recipes
|
||||
- title by author
|
||||
}}}
|
||||
|
||||
{{{ 5.35.0 2022-01-21
|
||||
|
||||
:: new features
|
||||
|
||||
- [1956006] Coloring/icon rules: Allow creating a rule for date columns that matches *today*
|
||||
|
||||
- Kobo driver: Add support for new firmware
|
||||
|
||||
- [1954890] Content server: Show total number of results when searching for books
|
||||
|
||||
:: bug fixes
|
||||
|
||||
- [1958028] E-book viewer: Fix searching for text near the end of a chapter sometimes not working
|
||||
|
||||
- [1954714] E-book viewer: Fix auto hyphenation on macOS not rendering the hyphens correctly
|
||||
|
||||
- Edit book: Reports: Fix thumbnails of SVG images not rendered
|
||||
|
||||
- ODT metadata: Support reading tags from multiple <keyword> elements
|
||||
|
||||
- [1958115] LRF Input: Fix a regression in calibre 5 that broke parsing of some LRF files
|
||||
|
||||
- [1956097] MOBI output: Dont fail if input document contains invalid % based lengths
|
||||
|
||||
- [1955308] AZW3 Input: Handle AZW3 files with incorrect TAGX Offset INDX header fields
|
||||
|
||||
- [1956932] Comic conversion: Fix conversion of comic images that are stored as grayscale images in JPEG format not working when converting to PDF with image processing turned off
|
||||
|
||||
- [1955967] calibredb catalog: Fix --ids and --search options not working for CSV/XML catalogs
|
||||
|
||||
- [1958490] Tag browser: Fix the find box not using all available width
|
||||
|
||||
- [1956192] E-book viewer: Remove books that do not exist from the recently opened book list
|
||||
|
||||
- Completion popups: Fix display of items containing line breaks
|
||||
|
||||
- [1956129] Fix line breaks in custom column descriptions not being rendered in their tooltips
|
||||
|
||||
- [1956088] Fix Preferences->Searching->Clear search histories not taking effect till a restart for some search boxes
|
||||
|
||||
- [1955732] Hierarchical entries in user category may not merge correctly in tag browser
|
||||
|
||||
:: improved recipes
|
||||
- Foreign Affairs
|
||||
- MIT Technology Review
|
||||
- Reuters
|
||||
- Clarin
|
||||
- General Knowledge Today
|
||||
- Popular Science
|
||||
|
||||
:: new recipes
|
||||
- Dw.de by xav
|
||||
- Equestria Daily by Timothee Andres
|
||||
}}}
|
||||
|
||||
{{{ 5.34.0 2021-12-17
|
||||
|
||||
:: new features
|
||||
|
||||
- Happy holidays to everyone!
|
||||
|
||||
- Driver for the new Nook Glowlight 4
|
||||
|
||||
- Edit book: Spell check tool: Add an exclude files button to exclude some files from being checked
|
||||
|
||||
- EPUB/MOBI Catalogs: Increase the maximum thumbnail size to 3 inches from 2 inches
|
||||
|
||||
- [1953739] Allow creating a shortcut in Preferences->Shortcuts->Edit metadata to paste metadata ignoring the value of the exclude_fields tweak
|
||||
|
||||
- [1954715] E-book viewer: Displays links marked up as glossary and bibliography links as popups
|
||||
|
||||
- [1954572] Add a tweak in Preferences->Tweaks to provide the sort value for undefined numbers
|
||||
|
||||
:: bug fixes
|
||||
|
||||
- Edit book: Fix pressing F8 to jump to next misspelled word not working after last word in current file
|
||||
|
||||
- [1954889] Fix PDB E-reader output broken in calibre 5
|
||||
|
||||
- [1954839] Edit book: Reports: Include descendant selectors that use classes when counting class usage
|
||||
|
||||
- [1954726] E-book viewer: Fix an error when opening some books with highlights that span in-line text formatting
|
||||
|
||||
- [1954460] MTP driver: Do not send the calibre device db files to the root folder on the Supernote A5 x as it fails
|
||||
|
||||
- ToC Editor: Workaround an occasional error when closing on Windows if the file being edited is in a DropBox/antivirus prone folder
|
||||
|
||||
- Fix a regression in the previous release that broke creating new keyboard shortcuts
|
||||
|
||||
- Comments editor: When flowing the tool bar onto multiple lines do not split up groups of buttons
|
||||
|
||||
- Various compatibility fixes for Python 3.10 used by some Linux distributions
|
||||
|
||||
:: improved recipes
|
||||
- Pocket
|
||||
- El Pais
|
||||
- American Prospect
|
||||
- Mediapart
|
||||
|
||||
}}}
|
||||
|
||||
{{{ 5.33.2 2021-12-03
|
||||
{{{ 5.33.0 2021-12-03
|
||||
|
||||
:: new features
|
||||
|
||||
|
|
@ -280,13 +53,10 @@
|
|||
|
||||
- [1951507] E-book viewer: Fix sorting of highlights incorrect in books that use HTML ids with a hyphen in them
|
||||
|
||||
- [1951467] PDF Output: Fix the option to break long words at the ends of lines causing boxes to be rendered at the end of the line on macOS with some fonts
|
||||
- [1951467] PDF Output: Fix the option to break long words at the ends of lines causing boxes to be rendered at the end fo the line on macOS with some fonts
|
||||
|
||||
- Google metadata plugin: When searching by ISBN if no results are found retry using an alternate query syntax
|
||||
|
||||
- 5.33.2 fixes a couple of regressions that broke the toolbar in the popup comments editor dialog and rendering of the download
|
||||
metadata button in the edit metadata dialog on Windows, as well as reading files from MTP devices on Windows
|
||||
|
||||
:: improved recipes
|
||||
- Smithsonian Magazine
|
||||
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ First create some empty top level directory and run the following commands::
|
|||
|
||||
git clone https://github.com/kovidgoyal/bypy.git
|
||||
git clone https://github.com/kovidgoyal/calibre.git
|
||||
cd bypy && git switch qt5 && cd ../calibre
|
||||
cd calibre
|
||||
|
||||
Now we need to bootstrap calibre, for which all its Linux build dependencies
|
||||
must have already been installed (see the `Dependencies
|
||||
|
|
|
|||
|
|
@ -253,7 +253,7 @@ def get_dependencies(self, path_to_lib):
|
|||
@flush
|
||||
def get_local_dependencies(self, path_to_lib):
|
||||
for x, is_id in self.get_dependencies(path_to_lib):
|
||||
if x.startswith('@rpath/Qt') or x.startswith('@rpath/libexpat'):
|
||||
if x.startswith('@rpath/Qt'):
|
||||
yield x, x[len('@rpath/'):], is_id
|
||||
elif x in ('libunrar.dylib', 'libstemmer.0.dylib', 'libstemmer.dylib') and not is_id:
|
||||
yield x, x, is_id
|
||||
|
|
|
|||
|
|
@ -626,8 +626,8 @@
|
|||
{
|
||||
"name": "feedparser",
|
||||
"unix": {
|
||||
"filename": "feedparser-6.0.8.tar.bz2",
|
||||
"hash": "sha256:5ce0410a05ab248c8c7cfca3a0ea2203968ee9ff4486067379af4827a59f9661",
|
||||
"filename": "feedparser-5.2.1.tar.bz2",
|
||||
"hash": "sha256:ce875495c90ebd74b179855449040003a1beb40cd13d5f037a0654251e260b02",
|
||||
"urls": ["pypi"]
|
||||
}
|
||||
},
|
||||
|
|
|
|||
|
|
@ -171,17 +171,14 @@ def sort_languages(x):
|
|||
return sort_key(type(u'')(name))
|
||||
|
||||
|
||||
website = 'https://calibre-ebook.com'
|
||||
html_context['other_languages'].sort(key=sort_languages)
|
||||
html_context['support_text'] = _('Support calibre')
|
||||
html_context['support_tooltip'] = _('Contribute to support calibre development')
|
||||
html_context['homepage_url'] = website
|
||||
html_context['homepage_url'] = 'https://calibre-ebook.com'
|
||||
if needs_localization:
|
||||
html_context['homepage_url'] = localize_website_link(html_context['homepage_url'])
|
||||
extlinks = {
|
||||
'website_base': (website, None),
|
||||
'website': (html_context['homepage_url'] + '/%s', None),
|
||||
'download_file': (f'{website}/downloads/%s', '%s'),
|
||||
}
|
||||
del sort_languages, get_language
|
||||
|
||||
|
|
@ -239,13 +236,11 @@ def sort_languages(x):
|
|||
# If false, no module index is generated.
|
||||
# latex_use_modindex = True
|
||||
|
||||
# we use lualatex as it is actively maintained and pdflatex and xelatex fail
|
||||
# to render smart quotes and dashes
|
||||
latex_engine = 'lualatex'
|
||||
latex_logo = 'resources/logo.png'
|
||||
latex_show_pagerefs = True
|
||||
latex_show_urls = 'footnote'
|
||||
latex_elements = {
|
||||
'papersize':'letterpaper',
|
||||
'fontenc':r'\usepackage[T2A,T1]{fontenc}',
|
||||
'preamble': r'\renewcommand{\pageautorefname}{%s}' % _('page'),
|
||||
}
|
||||
|
|
|
|||
|
|
@ -652,7 +652,7 @@ calibre can automatically convert ``.docx`` files created by Microsoft Word 2007
|
|||
newer. Just add the file to calibre and click convert.
|
||||
|
||||
.. note::
|
||||
There is a :download_file:`demo .docx file <demos/demo.docx>`
|
||||
There is a `demo .docx file <https://calibre-ebook.com/downloads/demos/demo.docx>`_
|
||||
that demonstrates the capabilities of the calibre conversion engine. Just
|
||||
download it and convert it to EPUB or AZW3 to see what calibre can do.
|
||||
|
||||
|
|
@ -671,8 +671,9 @@ produce clean HTML that will convert well. Note that Word produces really messy
|
|||
HTML, converting it can take a long time, so be patient. If you have a newer
|
||||
version of Word available, you can directly save it as .docx as well.
|
||||
|
||||
Another alternative is to use the free LibreOffice. Open your .doc file in
|
||||
LibreOffice and save it as .docx, which can be directly converted in calibre.
|
||||
Another alternative is to use the free OpenOffice. Open your .doc file in
|
||||
OpenOffice and save it in OpenOffice's format .odt. calibre can directly convert
|
||||
.odt files.
|
||||
|
||||
Convert TXT documents
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
@ -790,10 +791,10 @@ calibre will automatically convert this .cbc file into a e-book with a Table of
|
|||
EPUB advanced formatting demo
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Various advanced formatting for EPUB files is demonstrated in this :download_file:`demo file <demos/demo.epub>`.
|
||||
Various advanced formatting for EPUB files is demonstrated in this `demo file <https://calibre-ebook.com/downloads/demos/demo.epub>`_.
|
||||
The file was created from hand coded HTML using calibre and is meant to be used as a template for your own EPUB creation efforts.
|
||||
|
||||
The source HTML it was created from is available :download_file:`demo.zip <demos/demo.zip>`. The settings used to create the
|
||||
The source HTML it was created from is available `demo.zip <https://calibre-ebook.com/downloads/demos/demo.zip>`_. The settings used to create the
|
||||
EPUB from the ZIP file are::
|
||||
|
||||
ebook-convert demo.zip .epub -vv --authors "Kovid Goyal" --language en --level1-toc '//*[@class="title"]' --disable-font-rescaling --page-breaks-before / --no-default-epub-cover
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ the folder in which you created :file:`__init__.py`::
|
|||
are in :file:`/Applications/calibre.app/Contents/MacOS/`.
|
||||
|
||||
You can download the Hello World plugin from
|
||||
:download_file:`helloworld_plugin.zip`.
|
||||
`helloworld_plugin.zip <https://calibre-ebook.com/downloads/helloworld_plugin.zip>`_.
|
||||
|
||||
Every time you use calibre to convert a book, the plugin's :meth:`run` method will be called and the
|
||||
converted book will have its publisher set to "Hello World". This is a trivial plugin, lets move on to
|
||||
|
|
@ -54,7 +54,7 @@ This plugin will be spread over a few files (to keep the code clean). It will sh
|
|||
how to create elements in the calibre user interface and how to access
|
||||
and query the books database in calibre.
|
||||
|
||||
You can download this plugin from :download_file:`interface_demo_plugin.zip`
|
||||
You can download this plugin from `interface_demo_plugin.zip <https://calibre-ebook.com/downloads/interface_demo_plugin.zip>`_
|
||||
|
||||
.. _import_name_txt:
|
||||
|
||||
|
|
@ -185,7 +185,7 @@ Edit book plugins
|
|||
|
||||
Now let's change gears for a bit and look at creating a plugin to add tools to
|
||||
the calibre book editor. The plugin is available here:
|
||||
:download_file:`editor_demo_plugin.zip`.
|
||||
`editor_demo_plugin.zip <https://calibre-ebook.com/downloads/editor_demo_plugin.zip>`_.
|
||||
|
||||
The first step, as for all plugins is to create the
|
||||
import name empty txt file, as described :ref:`above <import_name_txt>`.
|
||||
|
|
@ -259,7 +259,7 @@ data in a separate process. A simple example plugin follows that shows how
|
|||
to do this.
|
||||
|
||||
You can download the plugin from
|
||||
:download_file:`webengine_demo_plugin.zip`.
|
||||
`webengine_demo_plugin.zip <https://calibre-ebook.com/downloads/webengine_demo_plugin.zip>`_.
|
||||
|
||||
The important part of the plugin is in two functions:
|
||||
|
||||
|
|
|
|||
|
|
@ -75,7 +75,7 @@ Getting the code
|
|||
------------------
|
||||
|
||||
You can get the calibre source code in two ways, using a version control system or
|
||||
directly downloading a :website_base:`tarball <dist/src>`.
|
||||
directly downloading a `tarball <https://calibre-ebook.com/dist/src>`_.
|
||||
|
||||
calibre uses `Git <https://www.git-scm.com/>`_, a distributed version control
|
||||
system. Git is available on all the platforms calibre supports. After
|
||||
|
|
@ -89,7 +89,7 @@ calibre is a very large project with a very long source control history, so the
|
|||
above can take a while (10 mins to an hour depending on your internet speed).
|
||||
|
||||
If you want to get the code faster, the source code for the latest release is
|
||||
always available as an :website_base:`archive <dist/src>`.
|
||||
always available as an `archive <https://calibre-ebook.com/dist/src>`_.
|
||||
|
||||
To update a branch to the latest code, use the command::
|
||||
|
||||
|
|
@ -210,9 +210,9 @@ Create a plain text file::
|
|||
export CALIBRE_DEVELOP_FROM="/Users/kovid/work/calibre/src"
|
||||
calibre-debug -g
|
||||
|
||||
Save this file as :file:`/usr/local/bin/calibre-develop`, then set its permissions so that it can be executed::
|
||||
Save this file as ``/usr/bin/calibre-develop``, then set its permissions so that it can be executed::
|
||||
|
||||
chmod +x /usr/local/bin/calibre-develop
|
||||
chmod +x /usr/bin/calibre-develop
|
||||
|
||||
Once you have done this, run::
|
||||
|
||||
|
|
@ -309,7 +309,7 @@ You can insert the following two lines of code to start an interactive Python se
|
|||
ipython(locals())
|
||||
|
||||
When running from the command line, this will start an interactive Python interpreter with access to all
|
||||
locally defined variables (variables in the local scope). The interactive prompt even has :kbd:`Tab` completion
|
||||
locally defined variables (variables in the local scope). The interactive prompt even has TAB completion
|
||||
for object properties and you can use the various Python facilities for introspection, such as
|
||||
:func:`dir`, :func:`type`, :func:`repr`, etc.
|
||||
|
||||
|
|
|
|||
|
|
@ -69,7 +69,7 @@ calibre is open source software while DRM by its very nature is closed. If
|
|||
calibre were to support opening or viewing DRM files it could be trivially
|
||||
modified to be used as a tool for DRM removal which is illegal under today's
|
||||
laws. Open source software and DRM are a clash of principles. While DRM is all
|
||||
about controlling the user, open source software is about empowering the user.
|
||||
about controlling the user open source software is about empowering the user.
|
||||
The two simply can not coexist.
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -26,7 +26,8 @@ Basic workflow
|
|||
---------------
|
||||
|
||||
.. note::
|
||||
A video tour of the calibre E-book editor is available :website:`here <demo#tutorials>`.
|
||||
A video tour of the calibre E-book editor is available `here
|
||||
<https://calibre-ebook.com/demo#tutorials>`_.
|
||||
|
||||
When you first open a book with the Edit book tool, you will be presented with
|
||||
a list of files on the left. These are the individual HTML files, stylesheets,
|
||||
|
|
@ -677,14 +678,6 @@ common in your book and to run a simple search and replace on individual words.
|
|||
check tool. If you do not do this and continue to use the Spell check tool,
|
||||
you could lose the changes you have made in the editor.
|
||||
|
||||
.. note::
|
||||
To exclude an individual file from being spell checked when running the
|
||||
spell check tool, you can use the :guilabel:`Exclude files` button or
|
||||
add the following comment just under the opening tag in the file::
|
||||
|
||||
<!-- calibre-no-spell-check -->
|
||||
|
||||
|
||||
Adding new dictionaries
|
||||
###########################
|
||||
|
||||
|
|
|
|||
|
|
@ -158,7 +158,8 @@ How do I use some of the advanced features of the conversion tools?
|
|||
You can get help on any individual feature of the converters by mousing over
|
||||
it in the GUI or running ``ebook-convert dummy.html .epub -h`` at a terminal.
|
||||
A good place to start is to look at the following demo file that demonstrates
|
||||
some of the advanced features :download_file:`html-demo.zip <html-demo.zip>`.
|
||||
some of the advanced features
|
||||
`html-demo.zip <https://calibre-ebook.com/downloads/html-demo.zip>`_
|
||||
|
||||
|
||||
Device integration
|
||||
|
|
@ -195,7 +196,7 @@ We just need some information from you:
|
|||
|
||||
Once you send us the output for a particular operating system, support for the device in that operating system
|
||||
will appear in the next release of calibre. To send us the output, open a bug report and attach the output to it.
|
||||
See :website:`how to report bugs <bugs>`.
|
||||
See `how to report bugs <https://calibre-ebook.com/bugs>`_.
|
||||
|
||||
My device is not being detected by calibre?
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
@ -215,8 +216,8 @@ Follow these steps to find the problem:
|
|||
fix that by looking under :guilabel:`System Preferences > Security and
|
||||
Privacy > Privacy > Files and Folders`.
|
||||
* Make sure you are running the latest version of calibre (currently
|
||||
|version|). The latest version can always be downloaded from :website:`the calibre
|
||||
website <download>`. You can tell what
|
||||
|version|). The latest version can always be downloaded from `the calibre
|
||||
website <https://calibre-ebook.com/download>`_. You can tell what
|
||||
version of calibre you are currently running by looking at the bottom
|
||||
line of the main calibre window.
|
||||
* Ensure your operating system is seeing the device. That is, the device
|
||||
|
|
@ -414,7 +415,7 @@ antivirus program.
|
|||
|
||||
.. note::
|
||||
If you are concerned about giving calibre access to your email
|
||||
account, simply create a new free email account with GMX or Outlook
|
||||
account, simply create a new free email account with GMX or Hotmail
|
||||
and use it only for calibre.
|
||||
|
||||
|
||||
|
|
@ -602,7 +603,7 @@ Why doesn't calibre have a column for foo?
|
|||
calibre is designed to have columns for the most frequently and widely used
|
||||
fields. In addition, you can add any columns you like. Columns can be added via
|
||||
:guilabel:`Preferences->Interface->Add your own columns`. Watch the tutorial
|
||||
:website:`UI Power tips <demo#tutorials>` to learn how to
|
||||
`UI Power tips <https://calibre-ebook.com/demo#tutorials>`_ to learn how to
|
||||
create your own columns, or read `this blog post
|
||||
<https://blog.calibre-ebook.com/calibre-custom-columns/>`_.
|
||||
|
||||
|
|
@ -920,13 +921,13 @@ Downloading from the Internet can sometimes result in a corrupted download. If t
|
|||
the :guilabel:`Security` tab. Make sure that your user account has full control
|
||||
for this folder.
|
||||
|
||||
If you still cannot get the installer to work and you are on Windows, you can use the :website:`calibre portable install <download_portable>`, which does not need an installer (it is just a ZIP file).
|
||||
If you still cannot get the installer to work and you are on Windows, you can use the `calibre portable install <https://calibre-ebook.com/download_portable>`_, which does not need an installer (it is just a zip file).
|
||||
|
||||
My antivirus program claims calibre is a virus/trojan?
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The first thing to check is that you are downloading calibre from the official
|
||||
website: :website:`<download>`. Make sure you are clicking the
|
||||
website: `<https://calibre-ebook.com/download>`_. Make sure you are clicking the
|
||||
download links on the left, not the advertisements on the right. calibre is a
|
||||
very popular program and unscrupulous people try to setup websites offering it
|
||||
for download to fool the unwary.
|
||||
|
|
@ -994,8 +995,8 @@ proxies used by calibre in Preferences->Miscellaneous.
|
|||
I want some feature added to calibre. What can I do?
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
You have two choices:
|
||||
1. Create a patch by hacking on calibre and send it to me for review and inclusion. See :website:`Development <get-involved>`.
|
||||
2. :website:`Open a bug requesting the feature <bugs>`. Remember that while you may think your feature request is extremely important/essential, calibre developers might not agree. Fortunately, calibre is open source, which means you always have the option of implementing your feature yourself, or hiring someone to do it for you. Furthermore, calibre has a comprehensive plugin architecture, so you might be able to develop your feature as a plugin, see :ref:`pluginstutorial`.
|
||||
1. Create a patch by hacking on calibre and send it to me for review and inclusion. See `Development <https://calibre-ebook.com/get-involved>`_.
|
||||
2. `Open a bug requesting the feature <https://calibre-ebook.com/bugs>`_. Remember that while you may think your feature request is extremely important/essential, calibre developers might not agree. Fortunately, calibre is open source, which means you always have the option of implementing your feature yourself, or hiring someone to do it for you. Furthermore, calibre has a comprehensive plugin architecture, so you might be able to develop your feature as a plugin, see :ref:`pluginstutorial`.
|
||||
|
||||
Why doesn't calibre have an automatic update?
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
@ -1007,8 +1008,8 @@ For many reasons:
|
|||
to see if you want to update once a year or so. There is a check box to
|
||||
turn off the update notification, on the update notification itself.
|
||||
|
||||
* calibre downloads currently use :website_base:`about 150TB of bandwidth a month
|
||||
<dynamic/downloads>`. Implementing automatic
|
||||
* calibre downloads currently use `about 150TB of bandwidth a month
|
||||
<https://calibre-ebook.com/dynamic/downloads>`_. Implementing automatic
|
||||
updates would greatly increase that and end up costing thousands of dollars
|
||||
a month, which someone has to pay.
|
||||
|
||||
|
|
@ -1031,7 +1032,7 @@ calibre is licensed under the GNU General Public License v3 (an open source lice
|
|||
How do I run calibre from my USB stick?
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
A portable version of calibre is available :website:`here <download_portable>`.
|
||||
A portable version of calibre is available `here <https://calibre-ebook.com/download_portable>`_.
|
||||
|
||||
How do I run parts of calibre like news download and the Content server on my own Linux server?
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
|
|||
|
|
@ -26,4 +26,4 @@ Glossary
|
|||
**URL** *(Uniform Resource Locator)* for example: ``http://example.com``
|
||||
|
||||
regexp
|
||||
**Regular expressions** provide a concise and flexible means for identifying strings of text of interest, such as particular characters, words, or patterns of characters. See :doc:`the tutorial <regexp>` for an introduction to regular expressions.
|
||||
**Regular expressions** provide a concise and flexible means for identifying strings of text of interest, such as particular characters, words, or patterns of characters. See `regexp syntax <https://docs.python.org/library/re.html>`_ for the syntax of regular expressions used in Python.
|
||||
|
|
|
|||
|
|
@ -495,7 +495,7 @@ be configured to read metadata from the file name instead, via
|
|||
:guilabel:`Preferences->Import/export->Adding books->Read metadata from file contents`.
|
||||
|
||||
You can also control how metadata is read from the filename using regular
|
||||
expressions (see :doc:`regexp`). In the :guilabel:`Adding books` section of
|
||||
expressions (see :doc:`regexp`). In the :guilabel:`Adding books` section of
|
||||
the configuration dialog, you can specify a regular expression that calibre
|
||||
will use to try and guess metadata from the names of e-book files that you add
|
||||
to the library. The default regular expression is::
|
||||
|
|
@ -566,7 +566,7 @@ The first click on an item will restrict the list of books to those that contain
|
|||
|
||||
Items in the Tag browser have their icons partially colored. The amount of color depends on the average rating of the books in that category. So for example if the books by Isaac Asimov have an average of four stars, the icon for Isaac Asimov in the Tag browser will be 4/5th colored. You can hover your mouse over the icon to see the average rating.
|
||||
|
||||
The outer-level items in the :guilabel:`Tag browser`, such as Authors and Series, are called categories. You can create your own categories, called :guilabel:`User categories`, which are useful for organizing items. For example, you can use the :guilabel:`User categories editor` (click the :guilabel:`Configure` button at the lower-left of the :guilabel:`Tag browser` and choose :guilabel:`Manage authors, tags, etc->User categories`) to create a User category called ``Favorite Authors``, then put the items for your favorites into the category. User categories can have sub-categories. For example, the User category ``Favorites.Authors`` is a sub-category of ``Favorites``. You might also have ``Favorites.Series``, in which case there will be two sub-categories under ``Favorites``. Sub-categories can be created by right-clicking on a User category, choosing :guilabel:`Add sub-category to...`, and entering the sub-category name; or by using the :guilabel:`User categories editor` by entering names like the Favorites example above.
|
||||
The outer-level items in the :guilabel:`Tag browser`, such as Authors and Series, are called categories. You can create your own categories, called :guilabel:`User categories`, which are useful for organizing items. For example, you can use the :guilabel:`User categories editor` (click the :guilabel:`Configure` button at the lower-left of the :guilabel:`Tag browser` and choose :guilabel:`Manage authors, series, etc->User categories`) to create a User category called ``Favorite Authors``, then put the items for your favorites into the category. User categories can have sub-categories. For example, the User category ``Favorites.Authors`` is a sub-category of ``Favorites``. You might also have ``Favorites.Series``, in which case there will be two sub-categories under ``Favorites``. Sub-categories can be created by right-clicking on a User category, choosing :guilabel:`Add sub-category to...`, and entering the sub-category name; or by using the :guilabel:`User categories editor` by entering names like the Favorites example above.
|
||||
|
||||
You can search User categories in the same way as built-in categories, by clicking on them. There are four different searches cycled through by clicking:
|
||||
1. "everything matching an item in the category" indicated by a single green plus sign.
|
||||
|
|
|
|||
|
|
@ -52,7 +52,7 @@ The normal edit metadata dialog also has :guilabel:`Next` and :guilabel:`Previou
|
|||
Search and replace
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The :guilabel:`Edit metadata for many books` dialog allows you to perform arbitrarily powerful search and replace operations on the selected books. By default it uses a simple text search and replace, but it also support *regular expressions*. For more on regular expressions, see :ref:`regexptutorial`.
|
||||
The :guilabel:`Bulk metadata edit` dialog allows you to perform arbitrarily powerful search and replace operations on the selected books. By default it uses a simple text search and replace, but it also support *regular expressions*. For more on regular expressions, see :ref:`regexptutorial`.
|
||||
|
||||
As noted above, there are two search and replace modes: character match and regular expression. Character match will look in the `Search field` you choose for the characters you type in the `search for` box and replace those characters with what you type in the `replace with` box. Each occurrence of the search characters in the field will be replaced. For example, assume the field being searched contains `a bad cat`. If you search for `a` to be replaced with `HELLO`, then the result will be `HELLO bHELLOd cHELLOt`.
|
||||
|
||||
|
|
|
|||
|
|
@ -105,7 +105,7 @@ This is Python, so indentation is important. After you've added the lines, it sh
|
|||
|
||||
In the above, ``def print_version(self, url)`` defines a *method* that is called by calibre for every article. ``url`` is the URL of the original article. What ``print_version`` does is take that url and replace it with the new URL that points to the print version of the article. To learn about `Python <https://www.python.org>`_ see the `tutorial <https://docs.python.org/tutorial/>`_.
|
||||
|
||||
Now, click the :guilabel:`Add/update recipe` button and your changes will be saved. Re-download the e-book. You should have a much improved e-book. One of the problems with the new version is that the fonts on the print version webpage are too small. This is automatically fixed when converting to an e-book, but even after the fixing process, the font size of the menus and navigation bar become too large relative to the article text. To fix this, we will do some more customization, in the next section.
|
||||
Now, click the :guilabel:`Add/update recipe` button and your changes will be saved. Re-download the e-book. You should have a much improved e-book. One of the problems with the new version is that the fonts on the print version webpage are too small. This is automatically fixed when converting to an e-book, but even after the fixing process, the font size of the menus and navigation bar to become too large relative to the article text. To fix this, we will do some more customization, in the next section.
|
||||
|
||||
Replacing article styles
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ You've just started calibre. What do you do now? Before calibre can do anything
|
|||
|
||||
Once you've admired the list of books you just added to your heart's content, you'll probably want to read one. In order to do that you'll have to convert the book to a format your reader understands. When first running calibre, the :guilabel:`Welcome wizard` starts and will set up calibre for your reader device. Conversion is a breeze. Just select the book you want to convert then click the "Convert books" button. Ignore all the options for now and click "OK". The little icon in the bottom right corner will start spinning. Once it's finished spinning, your converted book is ready. Click the "View" button to read the book.
|
||||
|
||||
If you want to read the book on your reader, connect the reader to the computer, wait till calibre detects it (10-20 seconds) and then click the "Send to device" button. Once the icon stops spinning again, disconnect your reader and read away! If you didn't convert the book in the previous step, calibre will auto convert it to the format your reader device understands.
|
||||
If you want to read the book on your reader, connect it to the computer, wait till calibre detects it (10-20 seconds) and then click the "Send to device" button. Once the icon stops spinning again, disconnect your reader and read away! If you didn't convert the book in the previous step, calibre will auto convert it to the format your reader device understands.
|
||||
|
||||
To get started with more advanced usage, you should read about :doc:`gui`. For even more power and versatility, learn the :doc:`generated/en/cli-index`. You will find the list of :doc:`faq` useful as well.
|
||||
|
||||
|
|
|
|||
|
|
@ -161,7 +161,7 @@ The functions intended for use in Single Function Mode are:
|
|||
* ``rating_to_stars(use_half_stars)`` -- Returns the rating as string of star (``★``) characters. The value must be a number between 0 and 5. Set use_half_stars to 1 if you want half star characters for fractional numbers available with custom ratings columns.
|
||||
* ``re(pattern, replacement)`` -- return the value after applying the regular expression. All instances of ``pattern`` in the value are replaced with ``replacement``. The template language uses case insensitive `Python regular expressions <https://docs.python.org/3/library/re.html>`_.
|
||||
* ``select(key)`` -- interpret the value as a comma-separated list of items with each item having the form ``id:value`` (the calibre ``identifier`` format). The function finds the first pair with the id equal to key and returns the corresponding value. If no id matches then the function returns the empty string.
|
||||
* ``shorten(left chars, middle text, right chars)`` -- Return a shortened version of the value, consisting of ``left chars`` characters from the beginning of the value, followed by ``middle text``, followed by ``right chars`` characters from the end of the value. ``Left chars`` and ``right chars`` must be non-negative integers. Example: assume you want to display the title with a length of at most 15 characters in length. One template that does this is ``{title:shorten(9,-,5)}``. For a book with the title `Ancient English Laws in the Times of Ivanhoe` the result will be `Ancient E-anhoe`: the first 9 characters of the title, a ``-``, then the last 5 characters. If the value's length is less than ``left chars`` + ``right chars`` + the length of ``middle text`` then the value will be returned unchanged. For example, the title `The Dome` would not be changed.
|
||||
* ``shorten(left chars, middle text, right chars)`` -- Return a shortened version of the value, consisting of ``left chars`` characters from the beginning of the value, followed by ``middle text``, followed by ``right chars`` characters from the end of the value. ``Left chars`` and ``right chars`` must be non-negative integers. Example: assume you want to display the title with a length of at most 15 characters in length. One template that does this is ``{title:shorten(9,-,5)}``. For a book with the title `Ancient English Laws in the Times of Ivanhoe` the result will be `Ancient E-nhoe`: the first 9 characters of the title, a ``-``, then the last 5 characters. If the value's length is less than ``left chars`` + ``right chars`` + the length of ``middle text`` then the value will be returned unchanged. For example, the title `The Dome` would not be changed.
|
||||
* ``str_in_list(separator, [ string, found_val, ]+ not_found_val)`` -- interpret the value as a list of items separated by ``separator`` then compare ``string`` against each value in the list. The ``string`` is not a regular expression. If ``string`` is equal to any item (ignoring case) then return the corresponding ``found_val``. If ``string`` contains ``separators`` then it is also treated as a list and each subvalue is checked. The ``string`` and ``found_value`` pairs can be repeated as many times as desired, permitting returning different values depending on string's value. If none of the strings match then ``not_found_value`` is returned. The strings are checked in order. The first match is returned.
|
||||
* ``subitems(start_index, end_index)`` -- This function breaks apart lists of tag-like hierarchical items such as genres. It interprets the value as a comma-separated list of tag-like items, where each item is a period-separated list. It returns a new list made by extracting from each item the components from ``start_index`` to ``end_index``, then merging the results back together. Duplicates are removed. The first subitem in a period-separated list has an index of zero. If an index is negative then it counts from the end of the list. As a special case, an end_index of zero is assumed to be the length of the list.
|
||||
|
||||
|
|
@ -678,7 +678,7 @@ The same thing happens for authors, but using a different character for the cut,
|
|||
|
||||
Plugboards affect the metadata written into the book when it is saved to disk or written to the device. Plugboards do not affect the metadata used by ``save to disk`` and ``send to device`` to create the file names. Instead, file names are constructed using the templates entered on the appropriate preferences window.
|
||||
|
||||
Tips
|
||||
Tips:
|
||||
-----
|
||||
|
||||
* Use the Template Tester to test templates. Add the tester to the context menu for books in the library and/or give it a keyboard shortcut.
|
||||
|
|
|
|||
|
|
@ -41,7 +41,7 @@ You can access the viewer controls by either:
|
|||
The viewer has two modes, "paged" and "flow". In paged mode the book content
|
||||
is presented as pages, similar to a paper book. In flow mode the text is
|
||||
presented continuously, like in a web browser. You can switch between them
|
||||
using the viewer :guilabel:`Preferences` under :guilabel:`Page layout` or by pressing the
|
||||
using the viewer Preferences under :guilabel:`Page layout` or by pressing the
|
||||
:kbd:`Ctrl+M` key.
|
||||
|
||||
|
||||
|
|
@ -62,7 +62,7 @@ Table of Contents
|
|||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
If the book you are reading defines a Table of Contents, you can access it by
|
||||
pressing the :guilabel:`Table of Contents` button. This will bring up a list
|
||||
pressing the :guilabel:`Table of Contents` button. This will bring up a list
|
||||
of sections in the book. You can click on any of them to jump to that portion
|
||||
of the book.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,24 +1,45 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
from __future__ import unicode_literals
|
||||
from __future__ import with_statement, unicode_literals
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
class AdvancedUserRecipe1645564525(BasicNewsRecipe):
|
||||
title = 'Аргументы и Факты - aif.ru'
|
||||
__author__ = 'MrBeef12'
|
||||
oldest_article = 7
|
||||
max_articles_per_feed = 100
|
||||
auto_cleanup = True
|
||||
class AdvancedUserRecipe1592177429(BasicNewsRecipe):
|
||||
title = 'Аргументы и Факты'
|
||||
encoding = 'utf8'
|
||||
language = 'ru'
|
||||
oldest_article = 7
|
||||
max_articles_per_feed = 25
|
||||
verbose = 3
|
||||
|
||||
feeds = [
|
||||
('ПОЛИТИКА', 'https://aif.ru/rss/politics.php'),
|
||||
('ДЕНЬГИ', 'https://aif.ru/rss/money.php'),
|
||||
('ОБЩЕСТВО', 'https://aif.ru/rss/society.php'),
|
||||
('ЗДОРОВЬЕ', 'https://aif.ru/rss/health.php'),
|
||||
('КУЛЬТУРА', 'https://aif.ru/rss/culture.php'),
|
||||
('СПОРТ', 'https://aif.ru/rss/sport.php'),
|
||||
('АВТОМОБИЛИ', 'https://aif.ru/rss/automobile.php'),
|
||||
('НЕДВИЖИМОСТЬ', 'https://aif.ru/rss/realty.php'),
|
||||
feeds = [
|
||||
('AIF', 'https://www.aif.ru/rss/all.php'),
|
||||
]
|
||||
INDEX = 'https://www.aif.ru/rss/all.php'
|
||||
|
||||
def parse_index(self):
|
||||
feeds = []
|
||||
section_title = 'aif'
|
||||
articles = []
|
||||
soup = self.index_to_soup(self.INDEX)
|
||||
ii = 0
|
||||
for item in soup.findAll('item'):
|
||||
if ii < self.max_articles_per_feed:
|
||||
try:
|
||||
ii = ii + 1
|
||||
A = str(item)
|
||||
i = A.find(u'link')
|
||||
j = A.find(u'description')
|
||||
ZZ = item.find('description')
|
||||
ZZ1 = str(ZZ) # bs4.element.Tag to str
|
||||
ZZ2 = ZZ1[24:-19]
|
||||
AU = A[i:j]
|
||||
try:
|
||||
articles.append({'url':AU[6:-2], 'title':ZZ2})
|
||||
except Exception:
|
||||
pass
|
||||
except Exception:
|
||||
self.log("Exception handled!")
|
||||
if articles:
|
||||
feeds.append((section_title, articles))
|
||||
return feeds
|
||||
|
|
|
|||
|
|
@ -4,7 +4,11 @@
|
|||
'''
|
||||
english.aljazeera.net
|
||||
'''
|
||||
from calibre.web.feeds.news import BasicNewsRecipe, classes
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
def has_cls(x):
|
||||
return dict(attrs={'class': lambda cls: cls and x in cls.split()})
|
||||
|
||||
|
||||
class AlJazeera(BasicNewsRecipe):
|
||||
|
|
@ -27,7 +31,7 @@ class AlJazeera(BasicNewsRecipe):
|
|||
'publisher': publisher, 'language': language
|
||||
}
|
||||
keep_only_tags = [
|
||||
classes('article-header article-featured-image wysiwyg--all-content'),
|
||||
dict(id='article-page'),
|
||||
]
|
||||
|
||||
remove_tags = [
|
||||
|
|
|
|||
|
|
@ -1,39 +1,20 @@
|
|||
#!/usr/bin/env python
|
||||
# License: GPLv3 Copyright: 2008, Kovid Goyal <kovid at kovidgoyal.net>
|
||||
|
||||
from __future__ import unicode_literals
|
||||
from calibre.web.feeds.news import BasicNewsRecipe, classes
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
class AmericanProspect(BasicNewsRecipe):
|
||||
title = 'American Prospect'
|
||||
__author__ = 'Kovid Goyal'
|
||||
oldest_article = 300
|
||||
title = u'American Prospect'
|
||||
__author__ = u'Michael Heinz, a.peter'
|
||||
version = 2
|
||||
|
||||
oldest_article = 30
|
||||
language = 'en'
|
||||
max_articles_per_feed = 100
|
||||
recursions = 0
|
||||
no_stylesheets = True
|
||||
remove_javascript = True
|
||||
encoding = 'utf-8'
|
||||
|
||||
use_embedded_content = False
|
||||
|
||||
no_stylesheets = True
|
||||
|
||||
keep_only_tags = [
|
||||
dict(id=['title', 'content']),
|
||||
]
|
||||
remove_tags = [
|
||||
classes('slideout-close-btn media-options')
|
||||
]
|
||||
|
||||
def get_feeds(self):
|
||||
soup = self.index_to_soup('https://prospect.org/archive')
|
||||
for a in soup.findAll('a', href=True):
|
||||
href = a['href']
|
||||
if href.endswith('-issue/'):
|
||||
d = href.strip('/').split('/')[-1]
|
||||
self.timefmt = ' [{}]'.format(d.rpartition('-')[0])
|
||||
self.log('Found magazine URL', href)
|
||||
return [('Articles', href + 'index.rss')]
|
||||
return [('Articles', 'https://prospect.org/api/rss/all.rss')]
|
||||
auto_cleanup = True
|
||||
feeds = [(u'Articles', u'feed://www.prospect.org/articles_rss.jsp')]
|
||||
|
|
|
|||
|
|
@ -111,23 +111,6 @@ def parse_article_json(root, abort_article):
|
|||
elif bt == 'text':
|
||||
lines.extend(serialize_text(block))
|
||||
return '<html><body id="main-content">' + '\n'.join(lines) + '</body></html>'
|
||||
|
||||
|
||||
def parse_raw_html(html, abort_article):
|
||||
q = '>window.__INITIAL_DATA__="{'
|
||||
idx = html.find(q)
|
||||
if idx < 0:
|
||||
raise ValueError('Failed to find JSON')
|
||||
data = html[idx + len(q) - 2:]
|
||||
idx = data.find('}";</script>')
|
||||
data = data[:idx+2]
|
||||
data = json.loads(data)
|
||||
root = json.loads(data)
|
||||
return parse_article_json(root, abort_article)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
print(parse_raw_html(open('/t/raw.html').read(), print))
|
||||
# }}}
|
||||
|
||||
|
||||
|
|
@ -286,4 +269,12 @@ class BBCNews(BasicNewsRecipe):
|
|||
resolve_internal_links = True
|
||||
|
||||
def preprocess_raw_html(self, raw_html, url):
|
||||
return parse_raw_html(raw_html, self.abort_article)
|
||||
q = '>window.__INITIAL_DATA__={'
|
||||
idx = raw_html.find(q)
|
||||
if idx < 0:
|
||||
raise ValueError('Failed to find JSON')
|
||||
data = raw_html[idx + len(q) - 1:]
|
||||
idx = data.find('};</script>')
|
||||
data = data[:idx+1]
|
||||
root = json.loads(data)
|
||||
return parse_article_json(root, self.abort_article)
|
||||
|
|
|
|||
|
|
@ -12,20 +12,18 @@
|
|||
def serialize_image(block):
|
||||
yield '<div>'
|
||||
block = block['model']
|
||||
img = block['image']
|
||||
alt = prepare_string_for_xml(img.get('alt') or '', True)
|
||||
media = block['media']
|
||||
alt = prepare_string_for_xml(media.get('alt') or '', True)
|
||||
for q in ('originalSrc', 'src'):
|
||||
if q in img:
|
||||
src = prepare_string_for_xml(img[q])
|
||||
if q in media:
|
||||
src = prepare_string_for_xml(media[q])
|
||||
break
|
||||
else:
|
||||
raise ValueError('No src found in img block: {}'.format(img))
|
||||
raise ValueError('No src found in media block: {}'.format(media))
|
||||
yield '<img src="{}" alt="{}"/>'.format(src, alt)
|
||||
caption = block.get('caption')
|
||||
if caption and caption.get('type') == 'text':
|
||||
yield '<div>'
|
||||
yield from serialize_paragraph(caption)
|
||||
yield '</div>'
|
||||
if caption:
|
||||
yield '<div>{}</div>'.format(prepare_string_for_xml(caption))
|
||||
yield '</div>'
|
||||
|
||||
|
||||
|
|
@ -104,30 +102,13 @@ def parse_article_json(root, abort_article):
|
|||
lines.append('<h1>{}</h1>'.format(prepare_string_for_xml(article['headline'])))
|
||||
if article.get('contributor'):
|
||||
lines.extend(serialize_contributor(article['contributor']))
|
||||
for block in article['content']['model']['blocks']:
|
||||
for block in article['blocks']:
|
||||
bt = block.get('type')
|
||||
if bt == 'image':
|
||||
lines.extend(serialize_image(block))
|
||||
elif bt == 'text':
|
||||
lines.extend(serialize_text(block))
|
||||
return '<html><body id="main-content">' + '\n'.join(lines) + '</body></html>'
|
||||
|
||||
|
||||
def parse_raw_html(html, abort_article):
|
||||
q = '>window.__INITIAL_DATA__="{'
|
||||
idx = html.find(q)
|
||||
if idx < 0:
|
||||
raise ValueError('Failed to find JSON')
|
||||
data = html[idx + len(q) - 2:]
|
||||
idx = data.find('}";</script>')
|
||||
data = data[:idx+2]
|
||||
data = json.loads(data)
|
||||
root = json.loads(data)
|
||||
return parse_article_json(root, abort_article)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
print(parse_raw_html(open('/t/raw.html').read(), print))
|
||||
# }}}
|
||||
|
||||
|
||||
|
|
@ -143,13 +124,10 @@ class BBC(BasicNewsRecipe):
|
|||
publisher = 'BBC'
|
||||
category = 'news, UK, world'
|
||||
language = 'en_GB'
|
||||
masthead_url = 'https://news.bbcimg.co.uk/img/1_0_1/cream/hi/news/news-blocks.gif'
|
||||
conversion_options = {
|
||||
'comments': description, 'tags': category, 'language': language, 'publisher': publisher,
|
||||
}
|
||||
# Removes empty feeds - why keep them!?
|
||||
remove_empty_feeds = True
|
||||
ignore_duplicate_articles = {'title', 'url'}
|
||||
resolve_internal_links = True
|
||||
|
||||
feeds = [
|
||||
('Top Stories', 'https://feeds.bbci.co.uk/news/rss.xml'),
|
||||
|
|
@ -172,4 +150,12 @@ class BBC(BasicNewsRecipe):
|
|||
]
|
||||
|
||||
def preprocess_raw_html(self, raw_html, url):
|
||||
return parse_raw_html(raw_html, self.abort_article)
|
||||
q = '>window.__INITIAL_DATA__={'
|
||||
idx = raw_html.find(q)
|
||||
if idx < 0:
|
||||
raise ValueError('Failed to find JSON')
|
||||
data = raw_html[idx + len(q) - 1:]
|
||||
idx = data.find('};</script>')
|
||||
data = data[:idx+1]
|
||||
root = json.loads(data)
|
||||
return parse_article_json(root, self.abort_article)
|
||||
|
|
|
|||
|
|
@ -34,11 +34,14 @@ class Clarin(BasicNewsRecipe):
|
|||
needs_subscription = 'optional'
|
||||
INDEX = 'http://www.clarin.com'
|
||||
LOGIN = 'https://app-pase.clarin.com/pase-registracion/app/pase/ingresarNavegable?execution=e1s1'
|
||||
masthead_url = 'http://www.clarin.com/images/logo_clarin.svg'
|
||||
cover_url = strftime('http://tapas.clarin.com/tapa/%Y/%m/%d/%Y%m%d_thumb.jpg')
|
||||
|
||||
compress_news_images = True
|
||||
scale_news_images_to_device = True
|
||||
compress_news_images_max_size = 10 # kB
|
||||
scale_news_images = True
|
||||
handle_gzip = True
|
||||
|
||||
# To get all the data (images)
|
||||
auto_cleanup = False
|
||||
|
|
@ -129,7 +132,7 @@ def get_browser(self):
|
|||
return br
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
for img in soup.findAll(['img'], attrs={'data-big': True}):
|
||||
for img in soup.findAll(['img']):
|
||||
img['src'] = img['data-big']
|
||||
|
||||
for figCaption in soup.findAll(['figcaption']):
|
||||
|
|
|
|||
|
|
@ -1,4 +1,8 @@
|
|||
from calibre.web.feeds.news import BasicNewsRecipe, classes
|
||||
from __future__ import with_statement
|
||||
__license__ = 'GPL 3'
|
||||
__copyright__ = '2014, Amit <amitkp.ias@gmail.com>'
|
||||
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
class My_Feeds(BasicNewsRecipe):
|
||||
|
|
@ -12,15 +16,19 @@ class My_Feeds(BasicNewsRecipe):
|
|||
center_navbar = True
|
||||
use_embedded_content = False
|
||||
remove_empty_feeds = True
|
||||
keep_only_tags = [
|
||||
classes('detail-heading content-main news-basic-info news-banner news-detail-content')
|
||||
]
|
||||
remove_tags_before = dict(name='div', id='PageContent')
|
||||
remove_tags_after = [dict(name='div'), {'class': 'taxonomy-terms'}]
|
||||
remove_tags = [
|
||||
classes('add-comment btn hindi_detail_link single-news-letter'),
|
||||
dict(id=['comments', 'breadcrumb', 'node_related_stories']),
|
||||
dict(attrs={'class': ['commentCount', 'box']})
|
||||
]
|
||||
|
||||
feeds = [
|
||||
('All', 'https://www.downtoearth.org.in/rss/all'),
|
||||
(u'editor', u'http://www.downtoearth.org.in/taxonomy/term/20348/0/feed'),
|
||||
(u'cover story', u'http://www.downtoearth.org.in/taxonomy/term/20345/0/feed'),
|
||||
(u'special report', u'http://www.downtoearth.org.in/taxonomy/term/20384/0/feed'),
|
||||
(u'features', u'http://www.downtoearth.org.in/taxonomy/term/20350/0/feed'),
|
||||
(u'news', u'http://www.downtoearth.org.in/taxonomy/term/20366/0/feed'),
|
||||
(u'debate', u'http://www.downtoearth.org.in/taxonomy/term/20347/0/feed'),
|
||||
(u'natural disasters', u'http://www.downtoearth.org.in/taxonomy/term/20822/0/feed')
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,29 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
class AdvancedUserRecipe1398527969(BasicNewsRecipe):
|
||||
title = u'DW-Deutsch XXL'
|
||||
language = 'de_DE'
|
||||
__author__ = 'xav'
|
||||
oldest_article = 7
|
||||
max_articles_per_feed = 100
|
||||
auto_cleanup = True
|
||||
no_stylesheets = True
|
||||
use_embedded_content = False
|
||||
remove_javascript = True
|
||||
feeds = [(u'Nachrichten', u'http://rss.dw.com/xml/rss-de-news'),
|
||||
(u'Themen des Tages', u'http://rss.dw.com/xml/rss-de-top'),
|
||||
(u'Langsam gesprochene Nachrichten',
|
||||
u'https://rss.dw.com/rdf/DKfeed_lgn_de'),
|
||||
(u'Wissenschaft', u'https://rss.dw.com/xml/rss-de-wissenschaft'),
|
||||
(u'Wirtschaft', u'https://rss.dw.com/xml/rss-de-eco'),
|
||||
(u'Wort der Woche',
|
||||
u'https://rss.dw.com/xml/DKpodcast_wortderwoche_de'),
|
||||
(u'Deutschland entdecken',
|
||||
u'http://rss.dw.com/xml/rss-de-deutschlandentdecken')]
|
||||
|
||||
def print_version(self, url):
|
||||
target = url.rpartition('/')[2]
|
||||
print_url = 'https://www.dw-world.de/popups/popup_printcontent/' + target
|
||||
return print_url
|
||||
|
|
@ -42,13 +42,10 @@ class ElPais(BasicNewsRecipe):
|
|||
'articulo-apertura',
|
||||
'articulo__contenedor'
|
||||
]}),
|
||||
dict(name='div', attrs={'class': 'a_c',}),
|
||||
|
||||
]
|
||||
|
||||
remove_tags = [
|
||||
dict(
|
||||
attrs={'class': [
|
||||
dict(attrs={'class': [
|
||||
'sumario__interior',
|
||||
'articulo-trust',
|
||||
'compartir',
|
||||
|
|
@ -57,11 +54,7 @@ class ElPais(BasicNewsRecipe):
|
|||
'more_info',
|
||||
'articulo-apoyos',
|
||||
'top10',
|
||||
]
|
||||
},
|
||||
),
|
||||
dict(id='cta_id'),
|
||||
dict(name='svg'),
|
||||
]}),
|
||||
]
|
||||
|
||||
feeds = [
|
||||
|
|
|
|||
|
|
@ -1,81 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
from calibre.utils.date import parse_date, utcnow
|
||||
|
||||
|
||||
class AdvancedUserRecipe1639926896(BasicNewsRecipe):
|
||||
__author__ = "Aisteru"
|
||||
__copyright__ = "2021, Timothée Andres <timothee dot andres at gmail dot com>"
|
||||
__license__ = 'GNU General Public License v3 - http://www.gnu.org/copyleft/gpl.html'
|
||||
|
||||
title = "Equestria Daily"
|
||||
description = "Everything new in Equestria and beyond!"
|
||||
language = 'en_US'
|
||||
|
||||
# Max. supported by website: 50
|
||||
max_articles_per_feed = 30
|
||||
|
||||
compress_news_images = True
|
||||
compress_news_images_auto_size = 4
|
||||
no_stylesheets = True
|
||||
keep_only_tags = [{'name': 'div', 'class_': ['post', 'hentry']}]
|
||||
remove_tags = [{'name': 'div', 'class_': 'post-footer'}]
|
||||
extra_css = '.article_date { margin-left: 10px; }'
|
||||
|
||||
# Masthead image dimensions
|
||||
MI_WIDTH = 600
|
||||
MI_HEIGHT = 200
|
||||
|
||||
# To discard posts under a certain section, simply comment the whole line
|
||||
sections = [
|
||||
("Art", 'Art'),
|
||||
("News", 'News'),
|
||||
("Fics", 'Fanfiction'),
|
||||
("Media", 'Media'),
|
||||
("Comics", 'Comic'),
|
||||
("Community", 'Community'),
|
||||
("Editorial", 'Editorial'),
|
||||
]
|
||||
|
||||
def get_masthead_url(self):
|
||||
soup = self.index_to_soup('https://www.equestriadaily.com')
|
||||
img = soup.select_one('#header img')
|
||||
return img['src']
|
||||
|
||||
def parse_index(self):
|
||||
results = {}
|
||||
current_date = utcnow()
|
||||
|
||||
def clean_description(description):
|
||||
lines = description.split('\n')
|
||||
return '\n'.join([line.strip() for line in lines if len(line.strip()) > 0])
|
||||
|
||||
for (section_name, section_url_name) in self.sections:
|
||||
soup = self.index_to_soup(
|
||||
f'https://www.equestriadaily.com/search/label/{section_url_name}?max-results={self.max_articles_per_feed}')
|
||||
articles = soup.select('div.post.hentry')
|
||||
previous_post_date = current_date
|
||||
|
||||
for article in articles:
|
||||
article_entry = {}
|
||||
|
||||
header = article.select_one('h3 > a')
|
||||
article_entry['title'] = header.text
|
||||
article_entry['url'] = header['href']
|
||||
article_entry['date'] = article.select_one('span.post-timestamp').text.split('\n')[1]
|
||||
article_entry['description'] = clean_description(article.select_one('div.entry-content').text)
|
||||
article_entry['content'] = '' # Must be empty
|
||||
|
||||
post_date = previous_post_date
|
||||
|
||||
try:
|
||||
post_date = parse_date(article_entry['date'])
|
||||
previous_post_date = post_date
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if (current_date - post_date).days <= self.oldest_article:
|
||||
results.setdefault(section_name, []).append(article_entry)
|
||||
|
||||
return [(section, results[section]) for section in results]
|
||||
|
|
@ -19,48 +19,35 @@ class FE_India(BasicNewsRecipe):
|
|||
description = 'Financial news from India'
|
||||
publisher = 'The Indian Express Limited'
|
||||
category = 'news, politics, finances, India'
|
||||
oldest_article = 2
|
||||
oldest_article = 30
|
||||
max_articles_per_feed = 200
|
||||
no_stylesheets = True
|
||||
encoding = 'utf-8'
|
||||
use_embedded_content = False
|
||||
language = 'en_IN'
|
||||
remove_empty_feeds = True
|
||||
ignore_duplicate_articles = {'url'}
|
||||
publication_type = 'magazine'
|
||||
|
||||
conversion_options = {
|
||||
'comment': description, 'tags': category, 'publisher': publisher, 'language': language
|
||||
}
|
||||
|
||||
keep_only_tags = [classes('wp-block-post-title wp-block-post-excerpt ie-network-post-meta-wrapper wp-block-post-featured-image wp-block-post-content')]
|
||||
keep_only_tags = [classes('post-title place-line leftstory')]
|
||||
remove_attributes = ['width', 'height']
|
||||
|
||||
feeds = [
|
||||
# https://www.financialexpress.com/syndication/
|
||||
# Print feeds
|
||||
('Front Page','https://www.financialexpress.com/print/front-page/feed/'),
|
||||
('Corporate Markets','https://www.financialexpress.com/print/corporate-markets/feed/'),
|
||||
('Economy','https://www.financialexpress.com/print/economy-print/feed/'),
|
||||
('Opinion','https://www.financialexpress.com/print/edits-columns/feed/'),
|
||||
('personal Finance','https://www.financialexpress.com/print/personal-finance-print/feed/'),
|
||||
# ('Brandwagon', 'https://www.financialexpress.com/print/brandwagon/feed/'),
|
||||
# Other Feeds
|
||||
('Latest news', 'https://www.financialexpress.com/feed/'),
|
||||
('Economy', 'https://www.financialexpress.com/economy/feed/'),
|
||||
('Banking & finance', 'https://www.financialexpress.com/industry/banking-finance/feed/'),
|
||||
('Opinion', 'https://www.financialexpress.com/opinion/feed/'),
|
||||
('Editorial', 'https://www.financialexpress.com/editorial/feed/'),
|
||||
('Budget', 'https://www.financialexpress.com/budget/feed/'),
|
||||
('Industry', 'https://www.financialexpress.com/industry/feed/'),
|
||||
('Market', 'https://www.financialexpress.com/market/feed/'),
|
||||
('Jobs', 'https://www.financialexpress.com/jobs/feed/'),
|
||||
('SME', 'https://www.financialexpress.com/industry/sme/feed/'),
|
||||
('Mutual Funds', 'https://www.financialexpress.com/money/mutual-funds/feed/'),
|
||||
('Health','https://www.financialexpress.com/lifestyle/health/feed'),
|
||||
# ('Health Care','https://www.financialexpress.com/healthcare/feed'),
|
||||
('Science','https://www.financialexpress.com/lifestyle/science/feed'),
|
||||
('Infrastructure','https://www.financialexpress.com/infrastructure/feed'),
|
||||
('Money','https://www.financialexpress.com/money/feed'),
|
||||
('Banking & finance', 'https://www.financialexpress.com/industry/banking-finance/feed/'),
|
||||
('Companies', 'https://www.financialexpress.com/industry/companies/feed/'),
|
||||
('Jobs', 'https://www.financialexpress.com/industry/jobs/feed/'),
|
||||
('Tech', 'https://www.financialexpress.com/industry/tech/feed/'),
|
||||
('Lifestyle', 'https://www.financialexpress.com/industry/lifestyle/feed/'),
|
||||
('Health', 'https://www.financialexpress.com/industry/health/feed/'),
|
||||
('Science', 'https://www.financialexpress.com/industry/science/feed/'),
|
||||
('Sports', 'https://www.financialexpress.com/industry/sports/feed/'),
|
||||
('Fe Columnist', 'https://www.financialexpress.com/industry/fe-columnist/feed/'),
|
||||
]
|
||||
|
||||
def preprocess_html(self, soup, *a):
|
||||
|
|
|
|||
|
|
@ -136,7 +136,7 @@ class ForeignAffairsRecipe(BasicNewsRecipe):
|
|||
classes('article-header article-body article-lead-image article-body-text'),
|
||||
]
|
||||
remove_tags = [
|
||||
classes('loading-indicator paywall article-footer article-tools')
|
||||
classes('loading-indicator paywall article-footer')
|
||||
]
|
||||
|
||||
conversion_options = {'comments': description, 'tags': category, 'language': 'en',
|
||||
|
|
@ -149,12 +149,10 @@ def parse_index(self):
|
|||
soup.head.title.string))[0]
|
||||
self.title = "Foreign Affairs ({})".format(date)
|
||||
self.timefmt = u' [%s]' % date
|
||||
link = soup.find('link', rel='canonical', href=True)['href']
|
||||
link = soup.find('link', rel='revision', href=True)['href']
|
||||
year, volnum, issue_vol = link.split('/')[-3:]
|
||||
self.cover_url = soup.find(**classes('subscribe-callout-image'))['data-src'].split("|")[-1]
|
||||
self.cover_url = self.cover_url.split('?')[0]
|
||||
self.cover_url = self.cover_url.replace('.webp', '')
|
||||
self.cover_url = self.cover_url.replace('_webp_issue_small_2x', 'issue_large_2x')
|
||||
self.cover_url = (soup.find('img', {'class': 'subscribe-callout-image'})['data-src']
|
||||
.split("|", 1)[0].replace('issue_small_1x', 'issue_large_2x'))
|
||||
|
||||
cls = soup.find('body')['class']
|
||||
if isinstance(cls, (list, tuple)):
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
from calibre.web.feeds.news import BasicNewsRecipe, classes
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
class GKT(BasicNewsRecipe):
|
||||
|
|
@ -12,36 +12,33 @@ class GKT(BasicNewsRecipe):
|
|||
no_javascript = True
|
||||
auto_cleanup = True
|
||||
|
||||
def parse_gkt_section(self, url, ignore_error=False):
|
||||
try:
|
||||
root = self.index_to_soup(url, as_tree=True)
|
||||
except Exception:
|
||||
if ignore_error:
|
||||
return
|
||||
raise
|
||||
for a in root.xpath('//div[@class="posts-listing"]/h1/a[@href]'):
|
||||
title = self.tag_to_string(a).strip()
|
||||
url = a.get('href')
|
||||
if title and url:
|
||||
self.log('\tFound article:', title, 'at', url)
|
||||
yield {'title': title, 'url': url}
|
||||
|
||||
def parse_index(self):
|
||||
securl = 'https://www.gktoday.in/current-affairs/'
|
||||
ans = {}
|
||||
|
||||
def p_tags(h1):
|
||||
for sib in h1.next_siblings:
|
||||
if sib.name == 'h1':
|
||||
break
|
||||
if sib.name == 'p':
|
||||
yield sib
|
||||
|
||||
def find_cat(ps):
|
||||
for p in ps:
|
||||
for a in p.findAll('a', rel='tag'):
|
||||
return self.tag_to_string(a)
|
||||
|
||||
for i in range(1, 6):
|
||||
page = '' if i == 1 else 'page/' + str(i)
|
||||
self.log('Trying:', securl + page)
|
||||
soup = self.index_to_soup(securl + page)
|
||||
container = soup.find(**classes('left_middle_content'))
|
||||
for h1 in container.findAll('h1'):
|
||||
title = self.tag_to_string(h1)
|
||||
a = h1.find('a')
|
||||
if a is None:
|
||||
continue
|
||||
url = a['href']
|
||||
ps = tuple(p_tags(h1))
|
||||
category = find_cat(ps) or 'Unknown'
|
||||
ans.setdefault(category, []).append({
|
||||
'title': title, 'url': url, 'description': self.tag_to_string(ps[0])})
|
||||
self.log('\t' + title + ' ' + url)
|
||||
return list(ans.items())
|
||||
url = 'http://www.gktoday.in/'
|
||||
root = self.index_to_soup(url, as_tree=True)
|
||||
ans = []
|
||||
h3 = root.xpath('//h3[@class="widget-title"]')[1]
|
||||
for a in h3.getparent().xpath('descendant::li/a[@href]'):
|
||||
category = self.tag_to_string(a).strip()
|
||||
if 'PDF' in category or not category:
|
||||
continue
|
||||
url = a.get('href')
|
||||
self.log('Found section:', category, 'at', url)
|
||||
articles = list(self.parse_gkt_section(url)) + \
|
||||
list(self.parse_gkt_section(url + '/page/2', ignore_error=True))
|
||||
if articles:
|
||||
ans.append((category, articles))
|
||||
return ans
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ class TheHindu(BasicNewsRecipe):
|
|||
__author__ = 'Kovid Goyal'
|
||||
max_articles_per_feed = 100
|
||||
no_stylesheets = True
|
||||
remove_attributes = ['style', 'height', 'width']
|
||||
remove_attributes = ['style']
|
||||
extra_css = '.lead-img-cont { text-align: center; } ' \
|
||||
'.lead-img-caption { font-size: small; font-style: italic; } ' \
|
||||
'.mobile-author-cont { font-size: small; text-transform: uppercase; } ' \
|
||||
|
|
@ -37,9 +37,8 @@ class TheHindu(BasicNewsRecipe):
|
|||
]
|
||||
|
||||
def get_browser(self):
|
||||
br = BasicNewsRecipe.get_browser(self, user_agent='common_words/based')
|
||||
br = BasicNewsRecipe.get_browser(self)
|
||||
br.addheaders += [('Referer', self.epaper_url)] # needed for fetching cover
|
||||
# br.set_debug_http(True)
|
||||
return br
|
||||
|
||||
def get_cover_url(self):
|
||||
|
|
@ -55,8 +54,6 @@ def preprocess_html(self, soup):
|
|||
source.extract()
|
||||
except Exception:
|
||||
pass
|
||||
for img in soup.findAll(attrs={'data-original': True}):
|
||||
img['src'] = img['data-original']
|
||||
# Place intro beneath the title, skip duplicates
|
||||
try:
|
||||
soup.h1.insert_after(soup.find('h2', attrs={'class': 'intro'}))
|
||||
|
|
@ -79,7 +76,7 @@ def populate_article_metadata(self, article, soup, first):
|
|||
|
||||
def articles_from_soup(self, soup):
|
||||
ans = []
|
||||
div = soup.find('section', attrs={'id': 'section_1'})
|
||||
div = soup.find('section', attrs={'id': 'section_'})
|
||||
if div is None:
|
||||
return ans
|
||||
for ul in div.findAll('ul', attrs={'class': 'archive-list'}):
|
||||
|
|
@ -102,7 +99,7 @@ def parse_index(self):
|
|||
# {'title':'xxx', 'url':'http://www.thehindu.com/opinion/op-ed/rohingya-bangladeshs-burden-to-bear/article19694058.ece'},
|
||||
# {'title':'yyy', 'url':'http://www.thehindu.com/sci-tech/energy-and-environment/on-river-washed-antique-plains/article19699327.ece'}
|
||||
# ])]
|
||||
soup = self.index_to_soup('https://www.thehindu.com/todays-paper/')
|
||||
soup = self.index_to_soup('http://www.thehindu.com/todays-paper/')
|
||||
nav_div = soup.find(id='subnav-tpbar-latest')
|
||||
section_list = []
|
||||
|
||||
|
|
|
|||
|
|
@ -1,58 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
|
||||
from calibre.web.feeds.news import BasicNewsRecipe, classes
|
||||
|
||||
|
||||
class HindustanTimes(BasicNewsRecipe):
|
||||
title = u'Hindustan Times'
|
||||
description = 'News from India.'
|
||||
language = 'en_IN'
|
||||
__author__ = 'unkn0wn'
|
||||
oldest_article = 1 # days
|
||||
max_articles_per_feed = 50
|
||||
encoding = 'utf-8'
|
||||
use_embedded_content = False
|
||||
no_stylesheets = True
|
||||
remove_attributes = ['style', 'height', 'width']
|
||||
ignore_duplicate_articles = {'url'}
|
||||
extra_css = 'button { display: none; } '
|
||||
|
||||
keep_only_tags = [
|
||||
dict(name='h1'),
|
||||
dict(name='div', attrs={'class':'sortDec'}),
|
||||
dict(name='picture'),
|
||||
dict(name='figcaption'),
|
||||
classes('dateTime storyBy storyDetails detail freemiumText paywall'),
|
||||
]
|
||||
|
||||
remove_tags = [
|
||||
classes('htsWrapper shareArticle new__newsletter__signup signup__box subscribe freemium-card adMinHeight313'
|
||||
' storyTopics embed_div shareIcons close-btn'),
|
||||
dict(name='div', attrs={'class':[]}),
|
||||
dict(name='footer'),
|
||||
]
|
||||
|
||||
feeds = [
|
||||
('Editorial','https://www.hindustantimes.com/feeds/rss/editorials/rssfeed.xml'),
|
||||
('Opinion','https://www.hindustantimes.com/feeds/rss/opinion/rssfeed.xml'),
|
||||
('HT Insight','https://www.hindustantimes.com/feeds/rss/ht-insight/rssfeed.xml'),
|
||||
('Analysis','https://www.hindustantimes.com/feeds/rss/analysis/rssfeed.xml'),
|
||||
('India News','https://www.hindustantimes.com/feeds/rss/india-news/rssfeed.xml'),
|
||||
('World News','https://www.hindustantimes.com/feeds/rss/world-news/rssfeed.xml'),
|
||||
('Business','https://www.hindustantimes.com/feeds/rss/business/rssfeed.xml'),
|
||||
('Science','https://www.hindustantimes.com/feeds/rss/science/rssfeed.xml'),
|
||||
('Education','https://www.hindustantimes.com/feeds/rss/education/rssfeed.xml'),
|
||||
('Elections','https://www.hindustantimes.com/feeds/rss/elections/rssfeed.xml'),
|
||||
('Sports','https://www.hindustantimes.com/feeds/rss/sports/rssfeed.xml'),
|
||||
('Books','https://www.hindustantimes.com/feeds/rss/books/rssfeed.xml'),
|
||||
('HT Weekend','https://www.hindustantimes.com/feeds/rss/ht-weekend/rssfeed.xml'),
|
||||
# ('Entertainment','https://www.hindustantimes.com/feeds/rss/entertainment/rssfeed.xml'),
|
||||
# ('Lifestyle',''https://www.hindustantimes.com/feeds/rss/lifestyle/rssfeed.xml'),
|
||||
# ('Cities',''https://www.hindustantimes.com/feeds/rss/cities/rssfeed.xml'),
|
||||
# ('Budget',''https://www.hindustantimes.com/feeds/rss/budget/rssfeed.xml')
|
||||
]
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
for img in soup.findAll('img', attrs={'data-src': True}):
|
||||
img['src'] = img['data-src']
|
||||
return soup
|
||||
|
|
@ -1,40 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
|
||||
from calibre.web.feeds.news import BasicNewsRecipe, classes
|
||||
|
||||
|
||||
class IndiaLegalLive(BasicNewsRecipe):
|
||||
title = 'India Legal Magazine'
|
||||
language = 'en_IN'
|
||||
__author__ = 'unkn0wn'
|
||||
oldest_article = 7 # days
|
||||
max_articles_per_feed = 50
|
||||
encoding = 'utf-8'
|
||||
use_embedded_content = False
|
||||
no_stylesheets = True
|
||||
remove_attributes = ['style', 'height', 'width']
|
||||
|
||||
masthead_url = 'https://d2r2ijn7njrktv.cloudfront.net/IL/uploads/2020/12/03181846/india-legal-live-logo-218x73-1.png'
|
||||
|
||||
def get_cover_url(self):
|
||||
soup = self.index_to_soup('https://www.indialegallive.com/')
|
||||
for citem in soup.findAll('img', src=lambda s: s and s.endswith('shot.jpg')):
|
||||
return citem['src']
|
||||
|
||||
keep_only_tags = [
|
||||
dict(name='h1'),
|
||||
classes(
|
||||
'tdb_single_subtitle tdb_single_date tdb_single_featured_image tdb_single_content'
|
||||
),
|
||||
]
|
||||
remove_tags = [
|
||||
dict(name='div', attrs={'style':'position:absolute;top:0;left:-9999px;'}),
|
||||
]
|
||||
|
||||
feeds = [
|
||||
('Courts', 'https://www.indialegallive.com/constitutional-law-news/courts-news/rss'),
|
||||
('Ring Side', 'https://www.indialegallive.com/ringside/rss'),
|
||||
('Cover Story Articles', 'https://www.indialegallive.com/cover-story-articles/rss'),
|
||||
('Special', 'https://www.indialegallive.com/special/rss'),
|
||||
('Columns', 'https://www.indialegallive.com/column-news/rss'),
|
||||
]
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
from calibre.web.feeds.news import BasicNewsRecipe, classes
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
class IndiaToday(BasicNewsRecipe):
|
||||
|
|
@ -7,41 +7,14 @@ class IndiaToday(BasicNewsRecipe):
|
|||
__author__ = 'Krittika Goyal'
|
||||
oldest_article = 15 # days
|
||||
max_articles_per_feed = 25
|
||||
no_stylesheets = True
|
||||
use_embedded_content = False
|
||||
remove_attributes = ['style']
|
||||
|
||||
keep_only_tags = [
|
||||
dict(name='h1'),
|
||||
classes('story-kicker story-right'),
|
||||
dict(itemProp='articleBody'),
|
||||
]
|
||||
no_stylesheets = True
|
||||
auto_cleanup = True
|
||||
|
||||
feeds = [
|
||||
('Editor\'s Note','https://www.indiatoday.in/rss/1206516'),
|
||||
('Cover Story', 'https://www.indiatoday.in/rss/1206509'),
|
||||
('The Big Story', 'https://www.indiatoday.in/rss/1206614'),
|
||||
('UP Front','https://www.indiatoday.in/rss/1206609'),
|
||||
('Liesure','https://www.indiatoday.in/rss/1206551'),
|
||||
('Cover Story', 'https://www.indiatoday.in/rss/1206509'),
|
||||
('Nation', 'https://www.indiatoday.in/rss/1206514'),
|
||||
('Health','https://www.indiatoday.in/rss/1206515'),
|
||||
('Defence','https://www.indiatoday.in/rss/1206517'),
|
||||
('Guest Column','https://www.indiatoday.in/rss/1206612'),
|
||||
('States', 'https://www.indiatoday.in/rss/1206500'),
|
||||
('Economy', 'https://www.indiatoday.in/rss/1206513'),
|
||||
('Special Report','https://www.indiatoday.in/rss/1206616'),
|
||||
('Investigation','https://www.indiatoday.in/rss/1206617'),
|
||||
('Diplomacy','https://www.indiatoday.in/rss/1206512'),
|
||||
('Sports','https://www.indiatoday.in/rss/1206518'),
|
||||
]
|
||||
|
||||
def preprocess_raw_html(self, raw_html, url):
|
||||
from calibre.ebooks.BeautifulSoup import BeautifulSoup
|
||||
soup = BeautifulSoup(raw_html)
|
||||
for script in soup.findAll('script'):
|
||||
script.extract()
|
||||
for style in soup.findAll('style'):
|
||||
style.extract()
|
||||
for img in soup.findAll('img', attrs={'data-src': True}):
|
||||
img['src'] = img['data-src']
|
||||
return str(soup)
|
||||
|
|
|
|||
|
|
@ -17,31 +17,38 @@ class IndianExpress(BasicNewsRecipe):
|
|||
|
||||
no_stylesheets = True
|
||||
use_embedded_content = False
|
||||
remove_attributes = ['style','height','width']
|
||||
ignore_duplicate_articles = {'url'}
|
||||
|
||||
keep_only_tags = [
|
||||
classes('heading-part full-details')
|
||||
]
|
||||
remove_tags = [
|
||||
dict(name='div', attrs={'id':'ie_story_comments'}),
|
||||
dict(name='img', attrs={'src':'https://images.indianexpress.com/2021/06/explained-button-300-ie.jpeg'}),
|
||||
dict(name='a', attrs={'href':'https://indianexpress.com/section/explained/?utm_source=newbanner'}),
|
||||
dict(name='img', attrs={'src':'https://images.indianexpress.com/2021/06/opinion-button-300-ie.jpeg'}),
|
||||
dict(name='a', attrs={'href':'https://indianexpress.com/section/opinion/?utm_source=newbanner'}),
|
||||
classes('share-social appstext story-tags ie-int-campign-ad ie-breadcrumb custom_read_button unitimg copyright')
|
||||
classes('share-social appstext story-tags')
|
||||
]
|
||||
feeds = [
|
||||
('Front Page', 'https://indianexpress.com/print/front-page/feed/'),
|
||||
('Op-Ed', 'http://indianexpress.com/section/opinion/feed/'),
|
||||
('Science & Technology', 'http://indianexpress.com/section/technology/feed/'),
|
||||
('Movie Reviews', 'https://indianexpress.com/section/entertainment/movie-review/feed/'),
|
||||
('Sunday Eye', 'https://indianexpress.com/print/eye/feed/'),
|
||||
('Explained', 'https://indianexpress.com/section/explained/feed/'),
|
||||
('Delhi Confidential', 'https://indianexpress.com/section/delhi-confidential/feed'),
|
||||
('Economy', 'https://indianexpress.com/print/economy/feed'),
|
||||
('Express Network', 'https://indianexpress.com/print/express-network/'),
|
||||
# Want to add more? go-to:https://indianexpress.com/syndication/
|
||||
('Front Page',
|
||||
'http://indianexpress.com/print/front-page/feed/'),
|
||||
('Editorials',
|
||||
'http://indianexpress.com/section/opinion/editorials/feed/'),
|
||||
('Crime',
|
||||
'http://indianexpress.com/section/india/crime/feed/'),
|
||||
('Cricket',
|
||||
'http://indianexpress.com/section/sports/cricket/feed/'),
|
||||
('Health',
|
||||
'http://www.indianexpress.com/lifestyle/health/feed/'),
|
||||
('Asia',
|
||||
'http://indianexpress.com/section/world/asia/'),
|
||||
('Politics',
|
||||
'http://indianexpress.com/section/india/politics/feed/'),
|
||||
('Mumbai',
|
||||
'http://www.indianexpress.com/cities/mumbai/feed/'),
|
||||
('Op-Ed',
|
||||
'http://indianexpress.com/section/opinion/feed/'),
|
||||
('Lifestyle',
|
||||
'http://indianexpress.com/section/lifestyle/feed/'),
|
||||
('Science & Technology',
|
||||
'http://indianexpress.com/section/technology/feed/'),
|
||||
('Bollywood',
|
||||
'http://indianexpress.com/section/entertainment/bollywood/feed/'),
|
||||
]
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
|
|
|
|||
|
|
@ -51,37 +51,26 @@ def get_browser(self):
|
|||
# To understand the signin logic read signin javascript from submit button from
|
||||
# https://www.irishtimes.com/signin
|
||||
|
||||
br = BasicNewsRecipe.get_browser(self, user_agent='curl/7.80.0')
|
||||
ip_data = json.loads(br.open('https://ipapi.co//json').read())
|
||||
br = BasicNewsRecipe.get_browser(self)
|
||||
url = 'https://www.irishtimes.com/signin'
|
||||
deviceid = str(uuid4()).replace('-', '')
|
||||
# Enable debug stuff?
|
||||
# br.set_debug_http(True)
|
||||
br.open(url).read()
|
||||
from pprint import pprint
|
||||
pprint(ip_data)
|
||||
br.set_cookie('IT_country', ip_data['country_code'], '.irishtimes.com')
|
||||
br.set_cookie('IT_eu', 'true' if ip_data['in_eu'] else 'false', '.irishtimes.com')
|
||||
rurl = 'https://www.irishtimes.com/auth-rest-api/v1/paywall/login'
|
||||
rq = Request(rurl, headers={
|
||||
'Accept': '*/*',
|
||||
'Accept-Language': 'en-US,en;q=0.5',
|
||||
'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
|
||||
'Origin': 'https://www.irishtimes.com',
|
||||
'Referer': url,
|
||||
'X-Requested-With': 'XMLHttpRequest',
|
||||
'sec-fetch-site': 'same-origin',
|
||||
'sec-fetch-dest': 'empty',
|
||||
'sec-fetch-mode': 'cors',
|
||||
}, data=urlencode({'username': self.username, 'password': self.password, 'deviceid':deviceid, 'persistent':'on', 'rid': ''}))
|
||||
}, data=urlencode({'username': self.username, 'password': self.password,'deviceid':deviceid, 'persistent':'on'}))
|
||||
|
||||
r = br.open(rq)
|
||||
raw = r.read()
|
||||
data = json.loads(raw)
|
||||
# print(data)
|
||||
if r.code != 200 or b'user_id' not in raw:
|
||||
pprint(data)
|
||||
raise ValueError('Failed to log in check username/password')
|
||||
|
||||
# Set cookie
|
||||
|
|
|
|||
|
|
@ -1,69 +1,58 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
__license__ = "GPL v3"
|
||||
__copyright__ = (
|
||||
"2008-2013, Darko Miletic <darko.miletic at gmail.com>. "
|
||||
"2022, Albert Aparicio Isarn <aaparicio at posteo.net>"
|
||||
)
|
||||
"""
|
||||
__license__ = 'GPL v3'
|
||||
__copyright__ = '2008-2013, Darko Miletic <darko.miletic at gmail.com>'
|
||||
'''
|
||||
japantimes.co.jp
|
||||
"""
|
||||
'''
|
||||
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
def classes(classes):
|
||||
q = frozenset(classes.split(' '))
|
||||
return dict(attrs={
|
||||
'class': lambda x: x and frozenset(x.split()).intersection(q)})
|
||||
|
||||
|
||||
class JapanTimes(BasicNewsRecipe):
|
||||
title = "The Japan Times"
|
||||
__author__ = "Albert Aparicio Isarn (original recipe by Darko Miletic)"
|
||||
description = (
|
||||
"The latest news from Japan Times, Japan's leading English-language daily newspaper"
|
||||
)
|
||||
language = "en_JP"
|
||||
category = "news, politics, japan"
|
||||
publisher = "The Japan Times"
|
||||
title = 'The Japan Times'
|
||||
__author__ = 'Darko Miletic'
|
||||
description = "Daily news and features on Japan from the most widely read English-language newspaper in Japan. Coverage includes national news, business news, sports news, commentary and features on living in Japan, entertainment, the arts, education and more." # noqa
|
||||
language = 'en_JP'
|
||||
category = 'news, politics, japan'
|
||||
publisher = 'The Japan Times'
|
||||
oldest_article = 2
|
||||
max_articles_per_feed = 150
|
||||
no_stylesheets = True
|
||||
remove_javascript = True
|
||||
use_embedded_content = False
|
||||
encoding = "utf8"
|
||||
publication_type = "newspaper"
|
||||
masthead_url = "https://cdn-japantimes.com/wp-content/themes/jt_theme/library/img/japantimes-logo-tagline.png"
|
||||
extra_css = "body{font-family: Geneva,Arial,Helvetica,sans-serif}"
|
||||
encoding = 'utf8'
|
||||
publication_type = 'newspaper'
|
||||
extra_css = 'body{font-family: Geneva,Arial,Helvetica,sans-serif}'
|
||||
|
||||
conversion_options = {
|
||||
"comment": description,
|
||||
"tags": category,
|
||||
"publisher": publisher,
|
||||
"language": language,
|
||||
'comment': description, 'tags': category, 'publisher': publisher, 'language': language
|
||||
}
|
||||
|
||||
remove_tags_before = {"name": "h1"}
|
||||
remove_tags_after = {"name": "ul", "attrs": {"class": "single-sns-area"}}
|
||||
keep_only_tags = [
|
||||
{"name": "div", "attrs": {"class": "padding_block"}},
|
||||
# {"name": "h5", "attrs": {"class": "writer", "role": "author"}},
|
||||
# {"name": "p", "attrs": {"class": "credit"}},
|
||||
]
|
||||
remove_tags_after = dict(name='div', attrs={'class': 'entry'}),
|
||||
keep_only_tags = [dict(name='div', attrs={'class': 'padding_block'})]
|
||||
remove_tags = [
|
||||
{"name": "div", "id": "no_js_blocker", "attrs": {"class": "padding_block"}},
|
||||
{"name": "div", "attrs": {"class": "single-upper-meta"}},
|
||||
{"name": "ul", "attrs": {"class": "single-sns-area"}},
|
||||
dict(name=['iframe', 'embed', 'object', 'base', 'form']), dict(attrs={'class': [
|
||||
'meta_extras', 'related_articles']}), dict(attrs={'id': 'content_footer_menu'}),
|
||||
dict(id='no_js_blocker'),
|
||||
classes('single-sns-area jt-related-stories'),
|
||||
]
|
||||
feeds = [
|
||||
(u"Top Stories", u"https://www.japantimes.co.jp/feed/topstories/"),
|
||||
(u"News", u"https://www.japantimes.co.jp/news/feed/"),
|
||||
(u"Opinion", u"https://www.japantimes.co.jp/opinion/feed/"),
|
||||
(u"Life", u"https://www.japantimes.co.jp/life/feed/"),
|
||||
(u"Community", u"https://www.japantimes.co.jp/community/feed/"),
|
||||
(u"Culture", u"https://www.japantimes.co.jp/culture/feed/"),
|
||||
(u"Sports", u"https://www.japantimes.co.jp/sports/feed/"),
|
||||
|
||||
(u'News', u'http://www.japantimes.co.jp/news/feed/'),
|
||||
(u'Opinion', u'http://www.japantimes.co.jp/opinion/feed/'),
|
||||
(u'Life', u'http://www.japantimes.co.jp/opinion/feed/'),
|
||||
(u'Community', u'http://www.japantimes.co.jp/community/feed/'),
|
||||
(u'Culture', u'http://www.japantimes.co.jp/culture/feed/'),
|
||||
(u'Sports', u'http://www.japantimes.co.jp/sports/feed/')
|
||||
]
|
||||
|
||||
def get_article_url(self, article):
|
||||
rurl = BasicNewsRecipe.get_article_url(self, article)
|
||||
return rurl.partition("?")[0]
|
||||
return rurl.partition('?')[0]
|
||||
|
||||
def preprocess_raw_html(self, raw, url):
|
||||
return "<html><head>" + raw[raw.find("</head>") :]
|
||||
return '<html><head>' + raw[raw.find('</head>'):]
|
||||
|
|
|
|||
|
|
@ -17,10 +17,10 @@ class JournalofHospitalMedicine(BasicNewsRecipe):
|
|||
|
||||
def get_browser(self):
|
||||
br = BasicNewsRecipe.get_browser(self)
|
||||
br.open('https://onlinelibrary.wiley.com/')
|
||||
br.open('http://www3.interscience.wiley.com/cgi-bin/home')
|
||||
br.select_form(nr=0)
|
||||
br['login'] = self.username
|
||||
br['password'] = self.password
|
||||
br['j_username'] = self.username
|
||||
br['j_password'] = self.password
|
||||
response = br.submit()
|
||||
raw = response.read()
|
||||
if '<h2>LOGGED IN</h2>' not in raw:
|
||||
|
|
@ -29,7 +29,7 @@ def get_browser(self):
|
|||
|
||||
# TO GET ARTICLE TOC
|
||||
def johm_get_index(self):
|
||||
return self.index_to_soup('https://shmpublications.onlinelibrary.wiley.com/toc/15535606/current')
|
||||
return self.index_to_soup('http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1553-5606/currentissue')
|
||||
|
||||
# To parse artice toc
|
||||
def parse_index(self):
|
||||
|
|
|
|||
|
|
@ -1,17 +1,181 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
from __future__ import unicode_literals
|
||||
|
||||
'''
|
||||
Lenta.ru
|
||||
'''
|
||||
|
||||
from calibre.web.feeds.feedparser import parse
|
||||
from calibre.ebooks.BeautifulSoup import Tag
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
import re
|
||||
|
||||
|
||||
class AdvancedUserRecipe1645563203(BasicNewsRecipe):
|
||||
title = 'Lenta.ru - Новости'
|
||||
__author__ = 'MrBeef12'
|
||||
oldest_article = 7
|
||||
def new_tag(soup, name, attrs=()):
|
||||
impl = getattr(soup, 'new_tag', None)
|
||||
if impl is not None:
|
||||
return impl(name, attrs=dict(attrs))
|
||||
return Tag(soup, name, attrs=attrs or None)
|
||||
|
||||
|
||||
class LentaRURecipe(BasicNewsRecipe):
|
||||
title = u'Lenta.ru: \u041d\u043e\u0432\u043e\u0441\u0442\u0438'
|
||||
__author__ = 'Nikolai Kotchetkov'
|
||||
publisher = 'lenta.ru'
|
||||
category = 'news, Russia'
|
||||
description = u'''\u0415\u0436\u0435\u0434\u043d\u0435\u0432\u043d\u0430\u044f
|
||||
\u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u0433\u0430\u0437\u0435\u0442\u0430.
|
||||
\u041d\u043e\u0432\u043e\u0441\u0442\u0438 \u0441\u043e
|
||||
\u0432\u0441\u0435\u0433\u043e \u043c\u0438\u0440\u0430 \u043d\u0430
|
||||
\u0440\u0443\u0441\u0441\u043a\u043e\u043c
|
||||
\u044f\u0437\u044b\u043a\u0435'''
|
||||
description = u'Ежедневная интернет-газета. Новости со всего мира на русском языке'
|
||||
oldest_article = 3
|
||||
max_articles_per_feed = 100
|
||||
auto_cleanup = True
|
||||
language = 'ru'
|
||||
|
||||
feeds = [
|
||||
('Lenta.ru - Новости России и мира сегодня', 'https://lenta.ru/rss/'),
|
||||
]
|
||||
masthead_url = u'http://img.lenta.ru/i/logowrambler.gif'
|
||||
cover_url = u'http://img.lenta.ru/i/logowrambler.gif'
|
||||
|
||||
# Add feed names if you want them to be sorted (feeds of this list appear
|
||||
# first)
|
||||
sortOrder = [u'_default', u'В России', u'б.СССР', u'В мире']
|
||||
|
||||
encoding = 'cp1251'
|
||||
language = 'ru'
|
||||
no_stylesheets = True
|
||||
remove_javascript = True
|
||||
recursions = 0
|
||||
|
||||
conversion_options = {
|
||||
'comment': description, 'tags': category, 'publisher': publisher, 'language': language
|
||||
}
|
||||
|
||||
keep_only_tags = [dict(name='td', attrs={'class': ['statya', 'content']})]
|
||||
|
||||
remove_tags_after = [dict(name='p', attrs={'class': 'links'}), dict(
|
||||
name='div', attrs={'id': 'readers-block'})]
|
||||
|
||||
remove_tags = [dict(name='table', attrs={'class': ['vrezka', 'content']}), dict(name='div', attrs={
|
||||
'class': 'b240'}), dict(name='div', attrs={'id': 'readers-block'}), dict(name='p', attrs={'class': 'links'})]
|
||||
|
||||
feeds = [u'http://lenta.ru/rss/']
|
||||
|
||||
extra_css = 'h1 {font-size: 1.2em; margin: 0em 0em 0em 0em;} h2 {font-size: 1.0em; margin: 0em 0em 0em 0em;} h3 {font-size: 0.8em; margin: 0em 0em 0em 0em;}' # noqa
|
||||
|
||||
def parse_index(self):
|
||||
try:
|
||||
feedData = parse(self.feeds[0])
|
||||
if not feedData:
|
||||
raise NotImplementedError
|
||||
self.log("parse_index: Feed loaded successfully.")
|
||||
|
||||
def get_virtual_feed_articles(feed):
|
||||
if feed in feeds:
|
||||
return feeds[feed][1]
|
||||
self.log("Adding new feed: ", feed)
|
||||
articles = []
|
||||
feeds[feed] = (feed, articles)
|
||||
return articles
|
||||
|
||||
feeds = {}
|
||||
|
||||
# Iterate feed items and distribute articles using tags
|
||||
for item in feedData.entries:
|
||||
link = item.get('link', '')
|
||||
title = item.get('title', '')
|
||||
if '' == link or '' == title:
|
||||
continue
|
||||
article = {'title': title, 'url': link, 'description': item.get(
|
||||
'description', ''), 'date': item.get('date', ''), 'content': ''}
|
||||
if not item.get('tags'):
|
||||
get_virtual_feed_articles('_default').append(article)
|
||||
continue
|
||||
for tag in item.tags:
|
||||
addedToDefault = False
|
||||
term = tag.get('term', '')
|
||||
if '' == term:
|
||||
if (not addedToDefault):
|
||||
get_virtual_feed_articles(
|
||||
'_default').append(article)
|
||||
continue
|
||||
get_virtual_feed_articles(term).append(article)
|
||||
|
||||
# Get feed list
|
||||
# Select sorted feeds first of all
|
||||
result = []
|
||||
for feedName in self.sortOrder:
|
||||
if (not feeds.get(feedName)):
|
||||
continue
|
||||
result.append(feeds[feedName])
|
||||
del feeds[feedName]
|
||||
result = result + feeds.values()
|
||||
|
||||
return result
|
||||
|
||||
except Exception as err:
|
||||
self.log(err)
|
||||
raise NotImplementedError
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
return self.adeify_images(soup)
|
||||
|
||||
def postprocess_html(self, soup, first_fetch):
|
||||
|
||||
contents = new_tag(soup, 'div')
|
||||
|
||||
# Extract tags with given attributes
|
||||
extractElements = {'div': [{'id': 'readers-block'}]}
|
||||
|
||||
# Remove all elements that were not extracted before
|
||||
for tag, attrs in extractElements.items():
|
||||
for attr in attrs:
|
||||
garbage = soup.findAll(tag, attr)
|
||||
if garbage:
|
||||
for pieceOfGarbage in garbage:
|
||||
pieceOfGarbage.extract()
|
||||
|
||||
# Find article text using header
|
||||
# and add all elements to contents
|
||||
element = soup.find({'h1': True, 'h2': True})
|
||||
if (element):
|
||||
element.name = 'h1'
|
||||
while element:
|
||||
nextElement = element.nextSibling
|
||||
element.extract()
|
||||
contents.insert(len(contents.contents), element)
|
||||
element = nextElement
|
||||
|
||||
# Place article date after header
|
||||
dates = soup.findAll(text=re.compile(
|
||||
r'\d{2}\.\d{2}\.\d{4}, \d{2}:\d{2}:\d{2}'))
|
||||
if dates:
|
||||
for date in dates:
|
||||
for string in date:
|
||||
parent = date.parent
|
||||
if (parent and isinstance(parent, Tag) and 'div' == parent.name and 'dt' == ''.join(parent['class'])):
|
||||
# Date div found
|
||||
parent.extract()
|
||||
parent[
|
||||
'style'] = 'font-size: 0.5em; color: gray; font-family: monospace;'
|
||||
contents.insert(1, parent)
|
||||
break
|
||||
|
||||
# Place article picture after date
|
||||
pic = soup.find('img')
|
||||
if pic:
|
||||
picDiv = new_tag(soup, 'div')
|
||||
picDiv['style'] = 'width: 100%; text-align: center;'
|
||||
pic.extract()
|
||||
picDiv.insert(0, pic)
|
||||
title = pic.get('title', None)
|
||||
if title:
|
||||
titleDiv = new_tag(soup, 'div')
|
||||
titleDiv['style'] = 'font-size: 0.5em;'
|
||||
titleDiv.insert(0, title)
|
||||
picDiv.insert(1, titleDiv)
|
||||
contents.insert(2, picDiv)
|
||||
|
||||
body = soup.find('td', {'class': ['statya', 'content']})
|
||||
if body:
|
||||
body.replaceWith(contents)
|
||||
|
||||
return soup
|
||||
|
|
|
|||
|
|
@ -1,81 +1,36 @@
|
|||
#!/usr/bin/env python
|
||||
|
||||
from calibre.web.feeds.news import BasicNewsRecipe, classes
|
||||
from datetime import date
|
||||
|
||||
is_saturday = date.today().weekday() == 5
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
class LiveMint(BasicNewsRecipe):
|
||||
title = u'Live Mint'
|
||||
description = 'Financial News from India.'
|
||||
language = 'en_IN'
|
||||
__author__ = 'Krittika Goyal'
|
||||
oldest_article = 1 # days
|
||||
max_articles_per_feed = 50
|
||||
encoding = 'utf-8'
|
||||
use_embedded_content = False
|
||||
|
||||
no_stylesheets = True
|
||||
remove_attributes = ['style', 'height', 'width']
|
||||
auto_cleanup = True
|
||||
|
||||
if is_saturday:
|
||||
keep_only_tags = [
|
||||
dict(name='h1'),
|
||||
dict(name='h2', attrs={'id':'story-summary-0'}),
|
||||
dict(name='picture'),
|
||||
dict(name='div', attrs={'class':'innerBanCaption'}),
|
||||
dict(name='div', attrs={'id':'date-display-before-content'}),
|
||||
dict(name='div', attrs={'class':'storyContent'}),
|
||||
]
|
||||
remove_tags = [
|
||||
classes(
|
||||
'sidebarAdv similarStoriesClass moreFromSecClass'
|
||||
)
|
||||
]
|
||||
feeds = [
|
||||
('News', 'https://lifestyle.livemint.com/rss/news'),
|
||||
('Food','https://lifestyle.livemint.com/rss/food'),
|
||||
('Fashion','https://lifestyle.livemint.com/rss/fashion'),
|
||||
('How to Lounge','https://lifestyle.livemint.com/rss/how-to-lounge'),
|
||||
('Smart Living','https://lifestyle.livemint.com/rss/smart-living'),
|
||||
]
|
||||
else:
|
||||
keep_only_tags = [
|
||||
dict(name='h1'),
|
||||
dict(name='picture'),
|
||||
dict(name='figcaption'),
|
||||
classes('articleInfo FirstEle summary highlights paywall'),
|
||||
]
|
||||
remove_tags = [
|
||||
classes(
|
||||
'trendingSimilarHeight moreNews mobAppDownload label msgError msgOk'
|
||||
)
|
||||
]
|
||||
|
||||
feeds = [
|
||||
('Companies', 'https://www.livemint.com/rss/companies'),
|
||||
('Opinion', 'https://www.livemint.com/rss/opinion'),
|
||||
('Money', 'https://www.livemint.com/rss/money'),
|
||||
('Economy', 'https://www.livemint.com/rss/economy/'),
|
||||
('Politics', 'https://www.livemint.com/rss/politics'),
|
||||
('Science', 'https://www.livemint.com/rss/science'),
|
||||
('Industry', 'https://www.livemint.com/rss/industry'),
|
||||
('Education', 'https://www.livemint.com/rss/education'),
|
||||
('Sports', 'https://www.livemint.com/rss/sports'),
|
||||
('Technology', 'https://www.livemint.com/rss/technology'),
|
||||
('News', 'https://www.livemint.com/rss/news'),
|
||||
('Mutual Funds', 'https://www.livemint.com/rss/Mutual Funds'),
|
||||
('Markets', 'https://www.livemint.com/rss/markets'),
|
||||
('AI', 'https://www.livemint.com/rss/AI'),
|
||||
('Insurance', 'https://www.livemint.com/rss/insurance'),
|
||||
('Budget', 'https://www.livemint.com/rss/budget'),
|
||||
('Elections', 'https://www.livemint.com/rss/elections'),
|
||||
]
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
for img in soup.findAll('img', attrs={'data-src': True}):
|
||||
img['src'] = img['data-src']
|
||||
if is_saturday:
|
||||
for img in soup.findAll('img', attrs={'data-img': True}):
|
||||
img['src'] = img['data-img']
|
||||
return soup
|
||||
feeds = [
|
||||
('News',
|
||||
'http://www.livemint.com/rss/news'),
|
||||
('Technology',
|
||||
'http://www.livemint.com/rss/technology'),
|
||||
('Companies',
|
||||
'http://www.livemint.com/rss/companies'),
|
||||
('Consumer',
|
||||
'http://www.livemint.com/rss/consumer'),
|
||||
('Opinion',
|
||||
'http://www.livemint.com/rss/opinion'),
|
||||
('Money',
|
||||
'http://www.livemint.com/rss/money'),
|
||||
('Industry',
|
||||
'http://www.livemint.com/rss/industry'),
|
||||
('Economy Politics',
|
||||
'http://www.livemint.com/rss/economy_politics'),
|
||||
('Lounge',
|
||||
'http://www.livemint.com/rss/lounge'),
|
||||
('Sports',
|
||||
'http://www.livemint.com/rss/sports'),
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,142 +1,64 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
__license__ = "GPL v3"
|
||||
__copyright__ = (
|
||||
"2010, Hiroshi Miura <miurahr@linux.com>. "
|
||||
"2021, Albert Aparicio Isarn <aaparicio at posteo.net>"
|
||||
)
|
||||
|
||||
"""
|
||||
www.mainichi.jp/english
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
__license__ = 'GPL v3'
|
||||
__copyright__ = '2010, Hiroshi Miura <miurahr@linux.com>'
|
||||
'''
|
||||
www.mainichi.jp
|
||||
'''
|
||||
|
||||
import re
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
class MainichiEnglishNews(BasicNewsRecipe):
|
||||
title = u"The Mainichi"
|
||||
__author__ = "Albert Aparicio Isarn (old version by Hiroshi Miura)"
|
||||
|
||||
description = "Japanese traditional newspaper Mainichi news in English"
|
||||
publisher = "Mainichi News"
|
||||
publication_type = "newspaper"
|
||||
category = "news, japan"
|
||||
language = "en_JP"
|
||||
|
||||
index = "http://mainichi.jp/english/"
|
||||
masthead_url = index + "images/themainichi.png"
|
||||
|
||||
title = u'The Mainichi'
|
||||
__author__ = 'Hiroshi Miura'
|
||||
oldest_article = 2
|
||||
max_articles_per_feed = 40
|
||||
no_stylesheets = True
|
||||
description = 'Japanese traditional newspaper Mainichi news in English'
|
||||
publisher = 'Mainichi News'
|
||||
category = 'news, japan'
|
||||
language = 'en_JP'
|
||||
index = 'http://mainichi.jp/english/english/index.html'
|
||||
remove_javascript = True
|
||||
masthead_url = 'http://mainichi.jp/english/images/themainichi.png'
|
||||
|
||||
remove_tags_before = {"id": "main-cont"}
|
||||
remove_tags_after = {"class": "main-text"}
|
||||
remove_tags = [{"name": "div", "id": "tools"}, {"name": "div", "class": "sub"}]
|
||||
remove_tags_before = {'class': "NewsTitle"}
|
||||
remove_tags_after = {'class': "NewsBody clr"}
|
||||
|
||||
def get_pickup_section(self, soup):
|
||||
# Topmost story
|
||||
top = soup.find("section", attrs={"class": "pickup section"})
|
||||
top_link = top.find("p", attrs={"class": "midashi"}).find("a")
|
||||
def parse_feeds(self):
|
||||
|
||||
try:
|
||||
top_date = (
|
||||
soup.find("div", attrs={"id": "main"})
|
||||
.find("div", attrs={"class": "date-box"})
|
||||
.find("p", attrs={"class": "date"})
|
||||
.string
|
||||
)
|
||||
feeds = BasicNewsRecipe.parse_feeds(self)
|
||||
|
||||
top_date_formatted = datetime.strptime(top_date, "%A, %B %d, %Y").strftime("%Y/%m/%d")
|
||||
except AttributeError:
|
||||
# If date not present, assume it is from today
|
||||
top_date_formatted = datetime.now().strftime("%Y/%m/%d")
|
||||
|
||||
top_description = top.find("p", attrs={"class": "txt"}).text
|
||||
|
||||
return [
|
||||
{
|
||||
"title": top_link.string,
|
||||
"date": top_date_formatted,
|
||||
"url": "https:" + top_link["href"],
|
||||
"description": top_description,
|
||||
}
|
||||
]
|
||||
|
||||
def retrieve_news_from_column(self, column):
|
||||
column_news = []
|
||||
|
||||
for item in column.findAll("li"):
|
||||
if item:
|
||||
itema = item.find("a")
|
||||
date_item = itema.find("p", attrs={"class": "date"})
|
||||
|
||||
column_news.append(
|
||||
{
|
||||
"title": itema.find("span").string,
|
||||
"date": date_item.string.strip("（）") if date_item else "",
|
||||
"url": "https:" + itema["href"],
|
||||
"description": "",
|
||||
}
|
||||
)
|
||||
|
||||
return column_news
|
||||
|
||||
def get_top_stories(self, soup):
|
||||
top_stories = self.get_pickup_section(soup)
|
||||
|
||||
news_section = soup.find("section", attrs={"class": "newslist"})
|
||||
top_news = news_section.find("div", attrs={"class": "main-box"}).find("ul")
|
||||
|
||||
top_stories.extend(self.retrieve_news_from_column(top_news))
|
||||
|
||||
return top_stories
|
||||
|
||||
def get_editor_picks(self, soup):
|
||||
editor_picks = []
|
||||
|
||||
news_section = soup.find("section", attrs={"class": "newslist"})
|
||||
news = news_section.find("div", attrs={"class": "sub-box"}).find("ul")
|
||||
|
||||
editor_picks.extend(self.retrieve_news_from_column(news))
|
||||
|
||||
return editor_picks
|
||||
|
||||
def get_section(self, section):
|
||||
soup = self.index_to_soup(self.index + section + "index.html")
|
||||
|
||||
section_news_items = self.get_pickup_section(soup)
|
||||
|
||||
news_columns = (
|
||||
soup.find("section", attrs={"class": "newslist section"})
|
||||
.find("div", attrs={"class": "col-set"})
|
||||
.find("ul")
|
||||
)
|
||||
|
||||
section_news_items.extend(self.retrieve_news_from_column(news_columns))
|
||||
|
||||
return section_news_items
|
||||
|
||||
def parse_index(self):
|
||||
soup = self.index_to_soup(self.index + "index.html")
|
||||
|
||||
feeds = [
|
||||
("Top Stories", self.get_top_stories(soup)),
|
||||
("Editor's Picks", self.get_editor_picks(soup)),
|
||||
# ("Latest Articles", self.get_section(self.index + "latest"+"index.html")),
|
||||
("Japan", self.get_section("japan")),
|
||||
("World", self.get_section("world")),
|
||||
("Business", self.get_section("business")),
|
||||
("Sports", self.get_section("sports")),
|
||||
("Science", self.get_section("science")),
|
||||
("Entertainment", self.get_section("entertainment")),
|
||||
("Opinion", self.get_section("opinion")),
|
||||
("Lifestyle", self.get_section("lifestyle")),
|
||||
("Obituaries", self.get_section("obituaries")),
|
||||
]
|
||||
for curfeed in feeds:
|
||||
delList = []
|
||||
for a, curarticle in enumerate(curfeed.articles):
|
||||
if re.search(r'pheedo.jp', curarticle.url):
|
||||
delList.append(curarticle)
|
||||
if re.search(r'rssad.jp', curarticle.url):
|
||||
delList.append(curarticle)
|
||||
if len(delList) > 0:
|
||||
for d in delList:
|
||||
index = curfeed.articles.index(d)
|
||||
curfeed.articles[index:index + 1] = []
|
||||
|
||||
return feeds
|
||||
|
||||
def parse_index(self):
|
||||
feeds = []
|
||||
soup = self.index_to_soup(self.index)
|
||||
for section in soup.findAll('section'):
|
||||
newsarticles = []
|
||||
section_name = 'news'
|
||||
hds = section.find('div', attrs={'class': 'CategoryHead clr'})
|
||||
if hds:
|
||||
section_item = hds.find('h1')
|
||||
if section_item:
|
||||
section_name = section_item.find('a').string
|
||||
items = section.find('ul', attrs={'class': 'MaiLink'})
|
||||
for item in items.findAll('li'):
|
||||
if item:
|
||||
itema = item.find('a')
|
||||
newsarticles.append({
|
||||
'title': itema.string, 'date': '', 'url': itema['href'], 'description': ''
|
||||
})
|
||||
feeds.append((section_name, newsarticles))
|
||||
return feeds
|
||||
|
|
|
|||
|
|
@ -45,12 +45,9 @@ class Mediapart(BasicNewsRecipe):
|
|||
keep_only_tags = [
|
||||
dict(name='h1'),
|
||||
dict(name='div', **classes('author')),
|
||||
classes('news__heading__top__intro news__body__center__article')
|
||||
]
|
||||
remove_tags = [
|
||||
classes('login-subscribe print-source_url'),
|
||||
dict(name='svg'),
|
||||
classes('introduction content-article')
|
||||
]
|
||||
remove_tags = [classes('login-subscribe print-source_url')]
|
||||
conversion_options = {'smarten_punctuation': True}
|
||||
|
||||
masthead_url = "https://raw.githubusercontent.com/lhoupert/calibre_contrib/main/mediapart_masthead.png"
|
||||
|
|
|
|||
|
|
@ -7,7 +7,8 @@
|
|||
'''
|
||||
technologyreview.com
|
||||
'''
|
||||
from calibre.web.feeds.news import BasicNewsRecipe, prefixed_classes
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
import re
|
||||
|
||||
|
||||
def absurl(x):
|
||||
|
|
@ -37,14 +38,26 @@ class MitTechnologyReview(BasicNewsRecipe):
|
|||
tags = 'news, technology, science'
|
||||
no_stylesheets = True
|
||||
|
||||
"""
|
||||
regex for class names
|
||||
"""
|
||||
articleHeaderRegex= '^.*contentHeader__wrapper.*$'
|
||||
editorLetterHeaderRegex = "^.*contentHeader--vertical__wrapper.*$"
|
||||
articleContentRegex = "^.*contentbody__wrapper.*$"
|
||||
imagePlaceHolderRegex = "^.*image__placeholder.*$"
|
||||
advertisementRegex = "^.*sliderAd__wrapper.*$"
|
||||
|
||||
keep_only_tags = [
|
||||
prefixed_classes('contentHeader contentArticleHeader contentBody')
|
||||
dict(name='header', attrs={'class': re.compile(editorLetterHeaderRegex, re.IGNORECASE)}),
|
||||
dict(name='header', attrs={'class': re.compile(articleHeaderRegex, re.IGNORECASE)}),
|
||||
dict(name='div', attrs={'class': re.compile(articleContentRegex, re.IGNORECASE)})
|
||||
]
|
||||
remove_tags = [
|
||||
dict(name="aside"),
|
||||
dict(name="svg"),
|
||||
dict(name="blockquote"),
|
||||
prefixed_classes('image__placeholder sliderAd__wrapper'),
|
||||
dict(name="img", attrs={'class': re.compile(imagePlaceHolderRegex, re.IGNORECASE)}),
|
||||
dict(name="div", attrs={'class': re.compile(advertisementRegex, re.IGNORECASE)}),
|
||||
]
|
||||
|
||||
def parse_index(self):
|
||||
|
|
|
|||
|
|
@ -1,48 +1,58 @@
|
|||
#!/usr/bin/env python
|
||||
# News source: https://www.rtvslo.si
|
||||
# License: GPLv3
|
||||
# Copyright: 2022, TadejS
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
__copyright__ = '2010, BlonG'
|
||||
'''
|
||||
www.rtvslo.si
|
||||
'''
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
class MMCRTV(BasicNewsRecipe):
|
||||
title = u'MMC RTV Slovenija'
|
||||
__author__ = u'TadejS'
|
||||
__author__ = u'BlonG'
|
||||
description = u"Prvi interaktivni multimedijski portal, MMC RTV Slovenija"
|
||||
oldest_article = 3
|
||||
max_articles_per_feed = 100
|
||||
max_articles_per_feed = 20
|
||||
language = 'sl'
|
||||
no_stylesheets = True
|
||||
use_embedded_content = False
|
||||
encoding = 'utf-8'
|
||||
publication_type = 'newspaper'
|
||||
|
||||
cover_url = 'https://img.rtvslo.si/_static/novi/logo/tvmmc-light-bg.png'
|
||||
cover_url = 'https://sites.google.com/site/javno2010/home/rtv_slo_cover.jpg'
|
||||
|
||||
extra_css = '''
|
||||
h1{font-family:Arial,Helvetica,sans-serif; font-weight:bold;font-size:large;}
|
||||
h2{font-family:Arial,Helvetica,sans-serif; font-weight:normal;font-size:small;}
|
||||
p{font-family:Arial,Helvetica,sans-serif;font-size:small;}
|
||||
body{font-family:Helvetica,Arial,sans-serif;font-size:small;}
|
||||
'''
|
||||
|
||||
def print_version(self, url):
|
||||
split_url = url.split("/")
|
||||
print_url = 'http://www.rtvslo.si/index.php?c_mod=news&op=print&id=' + \
|
||||
split_url[-1]
|
||||
return print_url
|
||||
|
||||
keep_only_tags = [
|
||||
dict(name='header', attrs={'class': 'article-header'}),
|
||||
dict(name='div', attrs={'class': 'article-body'}),
|
||||
]
|
||||
remove_tags=[
|
||||
dict(name='div', attrs={'class':'gallery-grid'}),
|
||||
dict(name='div', attrs={'class':'gallery'}),
|
||||
dict(name='div', attrs={'class':'exposed-article'}),
|
||||
dict(name='div', attrs={'class':'d-lg-none'}),
|
||||
dict(name='div', attrs={'class':'section-heading'}),
|
||||
dict(name='div', attrs={'class': 'title'}),
|
||||
dict(name='div', attrs={'id': 'newsbody'}),
|
||||
dict(name='div', attrs={'id': 'newsblocks'}),
|
||||
]
|
||||
# remove_tags=[
|
||||
# 40 dict(name='div', attrs={'id':'newsblocks'}),
|
||||
# ]
|
||||
|
||||
feeds = [
|
||||
(u'Slovenija', u'https://www.rtvslo.si/feeds/01.xml'),
|
||||
(u'Evropska unija', u'https://www.rtvslo.si/feeds/16.xml'),
|
||||
(u'Svet', u'https://www.rtvslo.si/feeds/02.xml'),
|
||||
(u'Gospodarstvo', u'https://www.rtvslo.si/feeds/04.xml'),
|
||||
(u'Okolje', u'https://www.rtvslo.si/feeds/12.xml'),
|
||||
(u'Znanost in tehnologija', u'https://www.rtvslo.si/feeds/09.xml'),
|
||||
(u'Kultura', u'https://www.rtvslo.si/feeds/05.xml'),
|
||||
(u'Šport', u'https://www.rtvslo.si/feeds/03.xml'),
|
||||
(u'Zabava', u'https://www.rtvslo.si/feeds/06.xml'),
|
||||
(u'Ture avanture', u'https://www.rtvslo.si/feeds/28.xml'),
|
||||
(u'Črna kronika', u'https://www.rtvslo.si/feeds/08.xml'),
|
||||
(u'Slovenija', u'http://www.rtvslo.si/feeds/01.xml'),
|
||||
(u'Svet', u'http://www.rtvslo.si/feeds/02.xml'),
|
||||
(u'Evropska unija', u'http://www.rtvslo.si/feeds/16.xml'),
|
||||
(u'Gospodarstvo', u'http://www.rtvslo.si/feeds/04.xml'),
|
||||
(u'\u010crna kronika', u'http://www.rtvslo.si/feeds/08.xml'),
|
||||
(u'Okolje', u'http://www.rtvslo.si/feeds/12.xml'),
|
||||
(u'Znanost in tehnologija', u'http://www.rtvslo.si/feeds/09.xml'),
|
||||
(u'Zabava', u'http://www.rtvslo.si/feeds/06.xml'),
|
||||
(u'Ture avanture', u'http://www.rtvslo.si/feeds/28.xml'),
|
||||
]
|
||||
|
||||
# def preprocess_html(self, soup):
|
||||
# newsblocks = soup.find('div',attrs = ['id':'newsblocks'])
|
||||
# soup.find('div', attrs = {'id':'newsbody'}).insert(-1, newsblocks)
|
||||
# return soup
|
||||
|
|
|
|||
|
|
@ -186,8 +186,7 @@ def parse_todays_page(self):
|
|||
soup = self.read_nyt_metadata()
|
||||
script = soup.findAll('script', text=lambda x: x and 'window.__preloadedData' in x)[0]
|
||||
script = type(u'')(script)
|
||||
json_data = script[script.find('{'):script.rfind(';')].strip().rstrip(';')
|
||||
data = json.loads(json_data.replace(':undefined', ':null'))['initialState']
|
||||
data = json.loads(script[script.find('{'):script.rfind(';')].strip().rstrip(';'))['initialState']
|
||||
containers, sections = {}, {}
|
||||
article_map = {}
|
||||
gc_pat = re.compile(r'groupings.(\d+).containers.(\d+)')
|
||||
|
|
|
|||
|
|
@ -186,8 +186,7 @@ def parse_todays_page(self):
|
|||
soup = self.read_nyt_metadata()
|
||||
script = soup.findAll('script', text=lambda x: x and 'window.__preloadedData' in x)[0]
|
||||
script = type(u'')(script)
|
||||
json_data = script[script.find('{'):script.rfind(';')].strip().rstrip(';')
|
||||
data = json.loads(json_data.replace(':undefined', ':null'))['initialState']
|
||||
data = json.loads(script[script.find('{'):script.rfind(';')].strip().rstrip(';'))['initialState']
|
||||
containers, sections = {}, {}
|
||||
article_map = {}
|
||||
gc_pat = re.compile(r'groupings.(\d+).containers.(\d+)')
|
||||
|
|
|
|||
|
|
@ -100,7 +100,7 @@ def process_image_block(lines, block, data):
|
|||
|
||||
|
||||
def json_to_html(raw):
|
||||
data = json.loads(raw.replace(':undefined', ':null'))
|
||||
data = json.loads(raw)
|
||||
data = data['initialState']
|
||||
article = next(iter(data.values()))
|
||||
body = data[article['sprinkledBody']['id']]
|
||||
|
|
|
|||
|
|
@ -1,48 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
|
||||
from calibre.web.feeds.news import BasicNewsRecipe, classes
|
||||
|
||||
|
||||
class OpenMagazine(BasicNewsRecipe):
|
||||
title = u'Open Magazine'
|
||||
description = 'The weekly current affairs and features magazine.'
|
||||
language = 'en_IN'
|
||||
__author__ = 'unkn0wn'
|
||||
oldest_article = 7 # days
|
||||
max_articles_per_feed = 50
|
||||
encoding = 'utf-8'
|
||||
use_embedded_content = False
|
||||
no_stylesheets = True
|
||||
remove_attributes = ['style', 'height', 'width']
|
||||
masthead_url = 'https://openthemagazine.com/wp-content/themes/open/images/logo.png'
|
||||
ignore_duplicate_articles = {'url'}
|
||||
extra_css = '[id^="caption-attachment"] {font-size: small;font-style: italic;}'
|
||||
|
||||
def get_cover_url(self):
|
||||
soup = self.index_to_soup('https://openthemagazine.com/magazine/')
|
||||
tag = soup.find(attrs={'class': 'mb-2 right-image'})
|
||||
if tag:
|
||||
self.cover_url = tag.find('img')['src']
|
||||
return super().get_cover_url()
|
||||
|
||||
keep_only_tags = [
|
||||
classes('post-data post-thumb post-meta post-excerp'),
|
||||
]
|
||||
|
||||
remove_tags = [
|
||||
classes('top-social menu-social post-author blurb-social button-links'),
|
||||
]
|
||||
|
||||
remove_tags_after = [
|
||||
classes('post-meta post-excerp'),
|
||||
]
|
||||
|
||||
feeds = [
|
||||
('Cover Story', 'https://openthemagazine.com/cover-story/feed/'),
|
||||
('Columns', 'https://openthemagazine.com/columns/feed'),
|
||||
('Essays', 'https://openthemagazine.com/essays/feed'),
|
||||
('Features', 'https://openthemagazine.com/features/feed'),
|
||||
('Books', 'https://openthemagazine.com/lounge/books/feed'),
|
||||
('Art & Culture', 'https://openthemagazine.com/art-culture/feed'),
|
||||
('Cinema', 'https://openthemagazine.com/cinema/feed'),
|
||||
]
|
||||
|
|
@ -23,18 +23,18 @@ class AdvancedUserRecipe1282101454(BasicNewsRecipe):
|
|||
ignore_duplicate_articles = {'url'}
|
||||
no_stylesheets = True
|
||||
keep_only_tags = [
|
||||
classes('Article-header Article-excerpt Article-author Article-thumbnail Article-bodyText'),
|
||||
classes('content-body article-header featured-img'),
|
||||
]
|
||||
|
||||
def parse_section_index(self, slug):
|
||||
soup = self.index_to_soup('https://www.popsci.com/{}/'.format(slug))
|
||||
main = soup.find(**classes('main-content'))
|
||||
for div in main.findAll(**classes('Post')):
|
||||
a = div.find('a', href=True, **classes('Post-link'))
|
||||
main = soup.find(**classes('main-module'))
|
||||
for div in main.findAll(**classes('main-item')):
|
||||
a = div.find('a', href=True, **classes('linkable'))
|
||||
url = a['href']
|
||||
title = self.tag_to_string(div.find(**classes('Post-title')))
|
||||
title = self.tag_to_string(a.find(**classes('title')))
|
||||
desc = ''
|
||||
dek = div.find(**classes('Post-excerpt'))
|
||||
dek = a.find(**classes('dek'))
|
||||
if dek is not None:
|
||||
desc = self.tag_to_string(dek)
|
||||
self.log(' ', title, url)
|
||||
|
|
|
|||
|
|
@ -9,7 +9,6 @@
|
|||
'''
|
||||
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
from polyglot.urllib import urlencode
|
||||
|
||||
|
||||
def classes(classes):
|
||||
|
|
@ -30,7 +29,6 @@ class PublicoPT(BasicNewsRecipe):
|
|||
language = 'pt'
|
||||
remove_empty_feeds = True
|
||||
extra_css = ' body{font-family: Arial,Helvetica,sans-serif } img{margin-bottom: 0.4em} '
|
||||
needs_subscription = True
|
||||
|
||||
keep_only_tags = [
|
||||
dict(id='story-content story-header'.split()),
|
||||
|
|
@ -55,23 +53,6 @@ class PublicoPT(BasicNewsRecipe):
|
|||
(u'Tecnologia', u'http://feeds.feedburner.com/PublicoTecnologia')
|
||||
]
|
||||
|
||||
def get_browser(self):
|
||||
br = BasicNewsRecipe.get_browser(self)
|
||||
if self.username is not None and self.password is not None:
|
||||
postdata = urlencode({
|
||||
'username': self.username,
|
||||
'password': self.password
|
||||
})
|
||||
br.open(
|
||||
'https://www.publico.pt/api/user/login',
|
||||
data=postdata,
|
||||
timeout=self.timeout
|
||||
)
|
||||
|
||||
br.set_handle_refresh(True)
|
||||
|
||||
return br
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
for img in soup.findAll('img', attrs={'data-media-viewer':True}):
|
||||
img['src'] = img['data-media-viewer']
|
||||
|
|
|
|||
|
|
@ -1,10 +1,17 @@
|
|||
"""
|
||||
Pocket Calibre Recipe v1.5
|
||||
Pocket Calibre Recipe v1.4
|
||||
"""
|
||||
from calibre import strftime
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
from string import Template
|
||||
import json
|
||||
import operator
|
||||
import re
|
||||
import tempfile
|
||||
try:
|
||||
from urllib.parse import urlencode
|
||||
except ImportError:
|
||||
from urllib import urlencode
|
||||
try:
|
||||
from urllib.error import HTTPError, URLError
|
||||
except ImportError:
|
||||
|
|
@ -32,7 +39,7 @@ class Pocket(BasicNewsRecipe):
|
|||
# Settings people change
|
||||
oldest_article = 7.0
|
||||
max_articles_per_feed = 50
|
||||
minimum_articles = 1
|
||||
minimum_articles = 10
|
||||
mark_as_read_after_dl = True # Set this to False for testing
|
||||
sort_method = 'oldest' # MUST be either 'oldest' or 'newest'
|
||||
# To filter by tag this needs to be a single tag in quotes; IE 'calibre'
|
||||
|
|
@ -42,7 +49,7 @@ class Pocket(BasicNewsRecipe):
|
|||
no_stylesheets = True
|
||||
use_embedded_content = False
|
||||
needs_subscription = True
|
||||
articles_are_obfuscated = False
|
||||
articles_are_obfuscated = True
|
||||
apikey = '19eg0e47pbT32z4793Tf021k99Afl889'
|
||||
index_url = u'https://getpocket.com'
|
||||
read_api_url = index_url + u'/v3/get'
|
||||
|
|
@ -111,7 +118,7 @@ def parse_index(self):
|
|||
'item_id': pocket_article[0],
|
||||
'title': pocket_article[1]['resolved_title'],
|
||||
'date': pocket_article[1]['time_updated'],
|
||||
'url': pocket_article[1]['resolved_url'],
|
||||
'url': u'{0}/a/read/{1}'.format(self.index_url, pocket_article[0]),
|
||||
'real_url': pocket_article[1]['resolved_url'],
|
||||
'description': pocket_article[1]['excerpt'],
|
||||
'sort': pocket_article[1]['sort_id']
|
||||
|
|
@ -119,6 +126,49 @@ def parse_index(self):
|
|||
self.articles = sorted(self.articles, key=operator.itemgetter('sort'))
|
||||
return [("My Pocket Articles for {0}".format(strftime('[%I:%M %p]')), self.articles)]
|
||||
|
||||
def get_textview(self, url):
|
||||
"""
|
||||
Since Pocket's v3 API they removed access to textview. They also
|
||||
redesigned their page to make it much harder to scrape their textview.
|
||||
We need to pull the article, retrieve the formcheck id, then use it
|
||||
to querty for the json version
|
||||
This function will break when pocket hates us
|
||||
"""
|
||||
ajax_url = self.index_url + u'/a/x/getArticle.php'
|
||||
soup = self.index_to_soup(url)
|
||||
fc_tag = soup.find('script', text=re.compile("formCheck"))
|
||||
fc_id = re.search(r"formCheck = \'([\d\w]+)\';", fc_tag).group(1)
|
||||
article_id = url.split("/")[-1]
|
||||
data = urlencode({'itemId': article_id, 'formCheck': fc_id})
|
||||
try:
|
||||
response = self.browser.open(ajax_url, data)
|
||||
except HTTPError as e:
|
||||
self.log.exception("unable to get textview {0}".format(e.info()))
|
||||
raise e
|
||||
return json.load(response)['article']
|
||||
|
||||
def get_obfuscated_article(self, url):
|
||||
"""
|
||||
Our get_textview returns parsed json so prettify it to something well
|
||||
parsed by calibre.
|
||||
"""
|
||||
article = self.get_textview(url)
|
||||
template = Template('<h1>$title</h1><div class="body">$body</div>')
|
||||
with tempfile.NamedTemporaryFile(delete=False) as tf:
|
||||
tmpbody = article['article']
|
||||
for img in article['images']:
|
||||
imgdiv = '<div id="RIL_IMG_{0}" class="RIL_IMG"></div>'.format(
|
||||
article['images'][img]['image_id'])
|
||||
imgtag = r'<img src="{0}" \>'.format(
|
||||
article['images'][img]['src'])
|
||||
tmpbody = tmpbody.replace(imgdiv, imgtag)
|
||||
|
||||
tf.write(template.safe_substitute(
|
||||
title=article['title'],
|
||||
body=tmpbody
|
||||
))
|
||||
return tf.name
|
||||
|
||||
def mark_as_read(self, mark_list):
|
||||
actions_list = []
|
||||
for article_id in mark_list:
|
||||
|
|
|
|||
|
|
@ -12,14 +12,14 @@
|
|||
# This imports the version bundled with Calibre
|
||||
import lxml
|
||||
from lxml.builder import E
|
||||
respekt_url = 'https://www.respekt.cz'
|
||||
respekt_url = 'http://www.respekt.cz'
|
||||
|
||||
|
||||
class respektRecipe(BasicNewsRecipe):
|
||||
__author__ = 'Tomáš Hnyk'
|
||||
publisher = u'Respekt Publishing a. s.'
|
||||
description = u'Articles from the print edition'
|
||||
title = u'Respekt Magazine — Print'
|
||||
title = u'Respekt Magazine – Print'
|
||||
encoding = 'utf-8'
|
||||
language = 'cs'
|
||||
delay = 0.001
|
||||
|
|
@ -74,10 +74,10 @@ def preprocess_raw_html(self, raw_html, url):
|
|||
return raw_html
|
||||
|
||||
def parse_index(self):
|
||||
raw1 = self.index_to_soup('https://www.respekt.cz/tydenik/', raw=True)
|
||||
raw1 = self.index_to_soup('http://www.respekt.cz/tydenik/', raw=True)
|
||||
root1 = lxml.html.fromstring(raw1)
|
||||
current_edition_url = root1.xpath("//div[@class='heroissue']/a")[0].items()[0][1]
|
||||
raw2 = self.index_to_soup('https://www.respekt.cz/' + current_edition_url, raw=True)
|
||||
raw2 = self.index_to_soup('http://www.respekt.cz/' + current_edition_url, raw=True)
|
||||
root2 = lxml.html.fromstring(raw2)
|
||||
self.cover_url = root2.xpath("//i[contains(@class, 'heroissue-cover')]")[0].get("data-src")
|
||||
# Fetch date
|
||||
|
|
|
|||
|
|
@ -4,18 +4,17 @@
|
|||
|
||||
from __future__ import absolute_import, division, print_function, unicode_literals
|
||||
|
||||
import json
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
from calibre.web.feeds.news import BasicNewsRecipe, classes
|
||||
|
||||
country = 'us'
|
||||
country_defs = {
|
||||
'us': ('www.reuters.com', {
|
||||
'Business': 'finance',
|
||||
'Markets': 'finance/markets',
|
||||
'World': 'world',
|
||||
'Business': 'business',
|
||||
'Markets': 'markets',
|
||||
'Tech': 'technology',
|
||||
'Sports': 'lifestyle/sports',
|
||||
'Wealth': 'markets/wealth',
|
||||
'Politics': 'politics',
|
||||
'Tech': 'news/technology',
|
||||
'Wealth': 'finance/wealth',
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -33,25 +32,6 @@ def matcher(x):
|
|||
return {'attrs': {'class': matcher}}
|
||||
|
||||
|
||||
def extract_article_list(raw):
|
||||
if isinstance(raw, bytes):
|
||||
raw = raw.decode('utf-8')
|
||||
# open('/t/raw.html', 'w').write(raw)
|
||||
idx = raw.index(';Fusion.globalContent={')
|
||||
d = raw[idx:]
|
||||
d = d[d.index('{'):]
|
||||
data = json.JSONDecoder().raw_decode(d)[0]
|
||||
# from pprint import pformat
|
||||
# print(pformat(data), file=open('/t/raw.py', 'w'))
|
||||
k = 'arcResult' if 'arcResult' in data else 'result'
|
||||
for article in data[k]['articles']:
|
||||
yield {'title': article['title'], 'description': article['description'], 'url': article['canonical_url']}
|
||||
|
||||
|
||||
# if __name__ == '__main__':
|
||||
# print(list(extract_article_list(open('/t/World News _ Latest Top Stories _ Reuters.html').read())))
|
||||
|
||||
|
||||
class Reuters(BasicNewsRecipe):
|
||||
title = 'Reuters'
|
||||
description = 'News from all over'
|
||||
|
|
@ -59,32 +39,29 @@ class Reuters(BasicNewsRecipe):
|
|||
language = 'en'
|
||||
|
||||
keep_only_tags = [
|
||||
prefixed_classes('article-header__heading___ article-header__author___ article-body__content___'),
|
||||
prefixed_classes('ArticlePage-article-header ArticlePage-article-body'),
|
||||
]
|
||||
remove_tags = [
|
||||
prefixed_classes(
|
||||
'context-widget__tabs___'
|
||||
' ArticleBody-read-time-and-social Slideshow-expand-button- TwoColumnsLayout-footer- RegistrationPrompt__container___'
|
||||
' SocialEmbed__inner___'
|
||||
),
|
||||
prefixed_classes('ArticleBody-read-time-and-social Slideshow-expand-button- TwoColumnsLayout-footer-'),
|
||||
dict(name=['button', 'link']),
|
||||
]
|
||||
remove_attributes = ['style']
|
||||
extra_css = '''
|
||||
img { max-width: 100%; }
|
||||
'''
|
||||
|
||||
def preprocess_html(self, soup, *a):
|
||||
for noscript in soup.findAll('noscript'):
|
||||
if noscript.findAll('img'):
|
||||
noscript.name = 'div'
|
||||
meta = soup.find(attrs={'name': "sailthru.image.full"})
|
||||
if meta is not None:
|
||||
url = meta['content']
|
||||
body = soup.find(**prefixed_classes('ArticlePage-article-body'))
|
||||
if body is not None:
|
||||
div = soup.new_tag('div')
|
||||
div.append(soup.new_tag('img', src=url))
|
||||
body.insert(0, div)
|
||||
return soup
|
||||
|
||||
def parse_index(self):
|
||||
base, sections = country_defs[country]
|
||||
ans = []
|
||||
|
||||
for section_title in sections:
|
||||
for section_title in sorted(sections):
|
||||
slug = sections[section_title]
|
||||
self.log(section_title)
|
||||
articles = list(self.parse_reuters_section(base, slug))
|
||||
|
|
@ -96,8 +73,15 @@ def parse_index(self):
|
|||
|
||||
def parse_reuters_section(self, base, slug):
|
||||
url = 'https://' + base + '/' + slug
|
||||
raw = self.index_to_soup(url, raw=True)
|
||||
for article in extract_article_list(raw):
|
||||
article['url'] = 'https://{}{}'.format(base, article['url'])
|
||||
yield article
|
||||
self.log('\t', article['title'], article['url'])
|
||||
try:
|
||||
soup = self.index_to_soup(url)
|
||||
except Exception:
|
||||
self.log.error('Failed to load Reuters section:', url)
|
||||
return
|
||||
for div in soup.findAll(**classes('news-headline-list')):
|
||||
h3 = div.find(**classes('story-title'))
|
||||
a = h3.parent
|
||||
title = self.tag_to_string(h3)
|
||||
url = 'https://{}{}'.format(base, a['href'])
|
||||
self.log('\t', title, url)
|
||||
yield {'title': title, 'url': url}
|
||||
|
|
|
|||
|
|
@ -1,25 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
from calibre.web.feeds.news import BasicNewsRecipe
|
||||
|
||||
|
||||
class RtRecipe(BasicNewsRecipe):
|
||||
title = 'RT на русском'
|
||||
__author__ = 'Vuizur'
|
||||
oldest_article = 7
|
||||
max_articles_per_feed = 100
|
||||
auto_cleanup = False
|
||||
language = 'ru'
|
||||
|
||||
remove_tags_before = dict(name='h1')
|
||||
remove_tags_after = dict(name='a', attrs={'class':'tags-trends__link'})
|
||||
remove_tags = [
|
||||
dict(name='div', attrs={'class':'error-on-page'}),
|
||||
dict(name='div', attrs={'class':'short-url'}),
|
||||
dict(name='div', attrs={'class':'follows-channel'}),
|
||||
dict(name='a', attrs={'class':'tags-trends__link'})
|
||||
]
|
||||
|
||||
feeds = [
|
||||
('RT на русском', 'https://russian.rt.com/rss'),
|
||||
]
|
||||
|
|
@ -4,7 +4,7 @@
|
|||
CATEGORIES = {
|
||||
'smart-news': 'Smart News',
|
||||
'history': 'History',
|
||||
'science-nature': 'Science',
|
||||
'science': 'Science',
|
||||
'innovation': 'Innovation',
|
||||
'arts-culture': 'Arts & Culture',
|
||||
'travel': 'Travel',
|
||||
|
|
@ -58,7 +58,7 @@ def parse_index(self):
|
|||
ans = []
|
||||
for slug, title in CATEGORIES.items():
|
||||
url = 'https://www.smithsonianmag.com/category/' + slug + '/'
|
||||
self.log('Parsing section:', title, 'at:', url)
|
||||
self.log('Parsing section:', title)
|
||||
articles = list(self.parse_section(url))
|
||||
if articles:
|
||||
ans.append((title, articles))
|
||||
|
|
|
|||
|
|
@ -31,24 +31,27 @@ class StraitsTimes(BasicNewsRecipe):
|
|||
'comments': description, 'tags': category, 'language': language, 'publisher': publisher
|
||||
}
|
||||
keep_only_tags = [
|
||||
classes('node-header node-subheadline group-byline-info group-updated-timestamp group-image-frame field-name-body article-content-rawhtml')
|
||||
classes('node-header node-subheadline group-byline-info group-updated-timestamp group-image-frame field-name-body')
|
||||
]
|
||||
remove_tags = [
|
||||
classes('st_telegram_boilerplate dropdown-menu ads'),
|
||||
classes('st_telegram_boilerplate'),
|
||||
dict(name='source'),
|
||||
]
|
||||
|
||||
feeds = [
|
||||
(u'World' , u'https://www.straitstimes.com/news/world/rss.xml')
|
||||
,(u'Business' , u'https://www.straitstimes.com/news/business/rss.xml')
|
||||
,(u'Life' , u'https://www.straitstimes.com/news/life/rss.xml')
|
||||
,(u'Tech' , u'https://www.straitstimes.com/news/tech/rss.xml')
|
||||
,(u'Opinion' , u'https://www.straitstimes.com/news/opinion/rss.xml')
|
||||
,(u'Life' , u'https://www.straitstimes.com/news/life/rss.xml')
|
||||
,(u'Singapore' , u'https://www.straitstimes.com/news/singapore/rss.xml')
|
||||
,(u'Asia' , u'https://www.straitstimes.com/news/asia/rss.xml')
|
||||
,(u'Multimedia' , u'https://www.straitstimes.com/news/multimedia/rss.xml')
|
||||
,(u'Sport' , u'https://www.straitstimes.com/news/sport/rss.xml')
|
||||
(u'Top of the News' , u'http://www.straitstimes.com/print/top-of-the-news/rss.xml')
|
||||
,(u'World' , u'http://www.straitstimes.com/print/world/rss.xml')
|
||||
,(u'Home' , u'http://www.straitstimes.com/print/home/rss.xml')
|
||||
,(u'Business' , u'http://www.straitstimes.com/print/business/rss.xml')
|
||||
,(u'Life' , u'http://www.straitstimes.com/print/life/rss.xml')
|
||||
,(u'Science' , u'http://www.straitstimes.com/print/science/rss.xml')
|
||||
,(u'Digital' , u'http://www.straitstimes.com/print/digital/rss.xml')
|
||||
,(u'Insight' , u'http://www.straitstimes.com/print/insight/rss.xml')
|
||||
,(u'Opinion' , u'http://www.straitstimes.com/print/opinion/rss.xml')
|
||||
,(u'Forum' , u'http://www.straitstimes.com/print/forum/rss.xml')
|
||||
,(u'Big Picture' , u'http://www.straitstimes.com/print/big-picture/rss.xml')
|
||||
,(u'Community' , u'http://www.straitstimes.com/print/community/rss.xml')
|
||||
,(u'Education' , u'http://www.straitstimes.com/print/education/rss.xml')
|
||||
]
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
|
|
|
|||
|
|
@ -1,44 +0,0 @@
|
|||
from calibre.web.feeds.news import BasicNewsRecipe, classes
|
||||
|
||||
|
||||
class SwarajyaMag(BasicNewsRecipe):
|
||||
title = u'Swarajya Magazine'
|
||||
__author__ = 'unkn0wn'
|
||||
description = 'Swarajya - a big tent for liberal right of centre discourse that reaches out, engages and caters to the new India.'
|
||||
language = 'en_IN'
|
||||
no_stylesheets = True
|
||||
remove_javascript = True
|
||||
use_embedded_content = False
|
||||
remove_attributes = ['height', 'width']
|
||||
encoding = 'utf-8'
|
||||
|
||||
keep_only_tags = [
|
||||
classes('_2PqtR _1sMRD ntw8h author-bio'),
|
||||
]
|
||||
|
||||
remove_tags = [
|
||||
classes('_JscD _2r17a'),
|
||||
]
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
for img in soup.findAll('img', attrs={'data-src': True}):
|
||||
img['src'] = img['data-src'].split('?')[0]
|
||||
return soup
|
||||
|
||||
def parse_index(self):
|
||||
soup = self.index_to_soup('https://swarajyamag.com/all-issues')
|
||||
a = soup.find('a', href=lambda x: x and x.startswith('/issue/'))
|
||||
url = a['href']
|
||||
self.log('Downloading issue:', url)
|
||||
self.cover_url = a.find('img', attrs={'data-src': True})['data-src']
|
||||
soup = self.index_to_soup('https://swarajyamag.com' + url)
|
||||
ans = []
|
||||
|
||||
for a in soup.findAll(**classes('_2eOQr')):
|
||||
url = a['href']
|
||||
if url.startswith('/'):
|
||||
url = 'https://swarajyamag.com' + url
|
||||
title = self.tag_to_string(a)
|
||||
self.log(title, ' at ', url)
|
||||
ans.append({'title': title, 'url': url})
|
||||
return [('Articles', ans)]
|
||||
|
|
@ -16,53 +16,29 @@ class Federalist(BasicNewsRecipe):
|
|||
title = 'The Federalist'
|
||||
__author__ = 'Kovid Goyal'
|
||||
language = 'en'
|
||||
oldest_article = 10
|
||||
oldest_article = 7
|
||||
max_articles_per_feed = 100
|
||||
no_stylesheets = True
|
||||
encoding = 'utf-8'
|
||||
use_embedded_content = False
|
||||
remove_attributes = ['xmlns', 'lang', 'style', 'width', 'height']
|
||||
|
||||
extra_css = """
|
||||
.shortbio,.article-excerpt{font-style: italic}
|
||||
.article-author-details,.article-author-description,.article-meta-author,.article-meta-date,.article-thumbnail-caption{font-size: small}
|
||||
"""
|
||||
extra_css = '''
|
||||
.shortbio { margin: 1em; padding: 1em; font-style: italic }
|
||||
'''
|
||||
|
||||
keep_only_tags = [
|
||||
classes(
|
||||
'title-lg article-thumbnail post-categories article-excerpt article-author-details'
|
||||
' article-meta-author article-meta-date article-content article-body shortbio entry-header'
|
||||
' byline-month byline-standard alpha-byline article-author-description article-author-details'),
|
||||
classes('entry-header'),
|
||||
classes('wp-post-image post-categories entry-content shortbio byline-month byline-standard alpha-byline'),
|
||||
]
|
||||
remove_tags = [
|
||||
dict(name=['meta', 'link']),
|
||||
classes('auth-ad article-share article-tags attachment-post-thumbnail attachment-author-bio'),
|
||||
classes('auth-ad attachment-post-thumbnail attachment-author-bio'),
|
||||
]
|
||||
|
||||
feeds = [
|
||||
('All', 'https://thefederalist.com/feed/'),
|
||||
('All', 'http://thefederalist.com/feed/'),
|
||||
]
|
||||
|
||||
# def parse_index(self):
|
||||
# return [('Articles', [
|
||||
# {
|
||||
# 'title': 'test',
|
||||
# 'url': 'https://thefederalist.com/2022/03/09/propaganda-press-wield-bidens-russia-blame-game-to-gaslight-americans-about-expensive-gas/'},
|
||||
# {
|
||||
# 'title': 'test2',
|
||||
# 'url': 'https://thefederalist.com/2022/03/10/white-house-will-blame-anyone-but-biden-for-februarys-7-9-inflation-jump/',
|
||||
# }
|
||||
# ])]
|
||||
|
||||
def preprocess_raw_html_(self, raw_html, url):
|
||||
soup = self.index_to_soup(raw_html)
|
||||
# this website puts article-thumbnail images inside article-body in
|
||||
# some articles and outside it in others, so we have to special case it
|
||||
for ab in soup.findAll(**classes('article-body')):
|
||||
for img in ab.findAll(**classes('article-thumbnail')):
|
||||
del img['class']
|
||||
return str(soup)
|
||||
|
||||
def preprocess_html(self, soup):
|
||||
for img in soup.findAll('img', attrs={'data-lazy-src': True}):
|
||||
img['src'] = img['data-lazy-src']
|
||||
|
|
@ -73,3 +49,9 @@ def preprocess_html(self, soup):
|
|||
img.extract()
|
||||
seen.add(src)
|
||||
return soup
|
||||
|
||||
# def parse_index(self):
|
||||
# return [('Articles', [
|
||||
# {'title':'img', 'url':'http://thefederalist.com/2018/05/09/venezuelas-economic-problems-caused-socialism-not-falling-oil-prices/'},
|
||||
# {'title':'xxx', 'url':'http://thefederalist.com/2018/05/04/fans-take-on-marvel-dc-and-the-comic-book-industrys-sjw-self-destruction/'},
|
||||
# ])]
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -38,9 +38,11 @@
|
|||
use_series_auto_increment_tweak_when_importing = False
|
||||
|
||||
#: Add separator after completing an author name
|
||||
# Set this if the completion separator should be appended to the end of the
|
||||
# completed text to automatically begin a new completion operation for authors.
|
||||
# It can be either True or False
|
||||
# Should the completion separator be append
|
||||
# to the end of the completed text to
|
||||
# automatically begin a new completion operation
|
||||
# for authors.
|
||||
# Can be either True or False
|
||||
authors_completer_append_separator = False
|
||||
|
||||
#: Author sort name algorithm
|
||||
|
|
@ -248,8 +250,6 @@
|
|||
# French
|
||||
'fra' : (r'Le\s+', r'La\s+', r"L'", u'L´', u'L’', r'Les\s+', r'Un\s+', r'Une\s+',
|
||||
r'Des\s+', r'De\s+La\s+', r'De\s+', r"D'", u'D´', u'L’'),
|
||||
# Polish
|
||||
'pol': (),
|
||||
# Italian
|
||||
'ita': ('Lo\\s+', 'Il\\s+', "L'", 'L´', 'La\\s+', 'Gli\\s+',
|
||||
'I\\s+', 'Le\\s+', 'Uno\\s+', 'Un\\s+', 'Una\\s+', "Un'",
|
||||
|
|
@ -426,6 +426,37 @@
|
|||
# Example: locale_for_sorting = 'nb' -- sort using Norwegian rules.
|
||||
locale_for_sorting = ''
|
||||
|
||||
#: Number of columns for custom metadata in the edit metadata dialog
|
||||
# Set whether to use one or two columns for custom metadata when editing
|
||||
# metadata one book at a time. If True, then the fields are laid out using two
|
||||
# columns. If False, one column is used.
|
||||
metadata_single_use_2_cols_for_custom_fields = True
|
||||
|
||||
#: Order of custom column(s) in edit metadata
|
||||
# Controls the order that custom columns are listed in edit metadata single
|
||||
# and bulk. The columns listed in the tweak are displayed first and in the
|
||||
# order provided. Any columns not listed are displayed after the listed ones,
|
||||
# in alphabetical order. Do note that this tweak does not change the size of
|
||||
# the edit widgets. Putting comments widgets in this list may result in some
|
||||
# odd widget spacing when using two-column mode.
|
||||
# Enter a comma-separated list of custom field lookup names, as in
|
||||
# metadata_edit_custom_column_order = ['#genre', '#mytags', '#etc']
|
||||
metadata_edit_custom_column_order = []
|
||||
|
||||
#: Edit metadata custom column label width and elision point
|
||||
# Set the width of custom column labels shown in the edit metadata dialogs.
|
||||
# If metadata_edit_elide_labels is True then labels wider than the width
|
||||
# will be elided, otherwise they will be word wrapped. The maximum width is
|
||||
# computed by multiplying the average width of characters in the font by the
|
||||
# appropriate number.
|
||||
# Set the elision point to 'middle' to put the ellipsis (…) in the middle of
|
||||
# the label, 'right' to put it at the right end of the label, and 'left' to
|
||||
# put it at the left end.
|
||||
metadata_edit_elide_labels = True
|
||||
metadata_edit_bulk_cc_label_length = 25
|
||||
metadata_edit_single_cc_label_length = 12
|
||||
metadata_edit_elision_point = 'right'
|
||||
|
||||
#: The number of seconds to wait before sending emails
|
||||
# The number of seconds to wait before sending emails when using a
|
||||
# public email server like GMX/Hotmail/Gmail. Default is: 5 minutes
|
||||
|
|
@ -435,7 +466,7 @@
|
|||
# to be public relays here. Any relay host ending with one of the suffixes
|
||||
# in the list below will be considered a public email server.
|
||||
public_smtp_relay_delay = 301
|
||||
public_smtp_relay_host_suffixes = ['gmail.com', 'live.com', 'gmx.com', 'outlook.com']
|
||||
public_smtp_relay_host_suffixes = ['gmail.com', 'live.com', 'gmx.com']
|
||||
|
||||
#: The maximum width and height for covers saved in the calibre library
|
||||
# All covers in the calibre library will be resized, preserving aspect ratio,
|
||||
|
|
@ -500,14 +531,7 @@
|
|||
# completions you will now have to press Tab to select one before pressing
|
||||
# Enter. Which technique you prefer will depend on the state of metadata in
|
||||
# your library and your personal editing style.
|
||||
#
|
||||
# If preselect_first_completion is False and you want Tab to accept what you
|
||||
# typed instead of the first completion then set tab_accepts_uncompleted_text
|
||||
# to True. If you do this then to select from the completions you must press
|
||||
# the Down or Up arrow keys. The tweak tab_accepts_uncompleted_text is ignored
|
||||
# if preselect_first_completion is True
|
||||
preselect_first_completion = False
|
||||
tab_accepts_uncompleted_text = False
|
||||
|
||||
#: Completion mode when editing authors/tags/series/etc.
|
||||
# By default, when completing items, calibre will show you all the candidates
|
||||
|
|
@ -558,8 +582,6 @@
|
|||
# Edit metadata->Copy metadata/Paste metadata actions. For example,
|
||||
# exclude_fields_on_paste = ['cover', 'timestamp', '#mycolumn']
|
||||
# to prevent pasting of the cover, Date and custom column, mycolumn.
|
||||
# You can also add a shortcut in Preferences->Shortcuts->Edit metadata
|
||||
# to paste metadata ignoring this tweak.
|
||||
exclude_fields_on_paste = []
|
||||
|
||||
#: Skip internet connected check
|
||||
|
|
@ -572,16 +594,3 @@
|
|||
# Sets the width of the tab stop in the template editor in "average characters".
|
||||
# For example, a value of 1 results in a space with the width of one average character.
|
||||
template_editor_tab_stop_width = 4
|
||||
|
||||
#: Value for undefined numbers when sorting
|
||||
# Sets the value to use for undefined numbers when sorting.
|
||||
# For example, the value -10 sorts undefined numbers as if they were set to -10.
|
||||
# Use 'maximum' for the largest possible number. Use 'minimum' for the smallest
|
||||
# possible number. Quotes are optional if entering a number.
|
||||
# Examples:
|
||||
# value_for_undefined_numbers_when_sorting = -100
|
||||
# value_for_undefined_numbers_when_sorting = '2'
|
||||
# value_for_undefined_numbers_when_sorting = -0.01
|
||||
# value_for_undefined_numbers_when_sorting = 'minimum'
|
||||
# value_for_undefined_numbers_when_sorting = 'maximum'
|
||||
value_for_undefined_numbers_when_sorting = 0
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -1,5 +1,4 @@
|
|||
SET UTF-8
|
||||
FLAG UTF-8
|
||||
TRY aeroinsctldumpbgfvhzóíjáqéñxyúükwAEROINSCTLDUMPBGFVHZÓÍJÁQÉÑXYÚÜKW
|
||||
REP 20
|
||||
REP ás az
|
||||
|
|
@ -106,22 +105,15 @@ PFX u e tras e[^s]
|
|||
PFX v Y 2
|
||||
PFX v 0 contra [^r]
|
||||
PFX v 0 contrar r
|
||||
PFX w Y 1
|
||||
PFX w 0 ex .
|
||||
SFX A Y 14
|
||||
SFX A Y 9
|
||||
SFX A r ción/S ar
|
||||
SFX A er ición/S [^cn]er
|
||||
SFX A er ición/S [^e]cer
|
||||
SFX A ecer ición/S ecer
|
||||
SFX A er ición/S [^o]ner
|
||||
SFX A ner sición/S oner
|
||||
SFX A r ción/S [^bc]ir
|
||||
SFX A bir pción/S ebir
|
||||
SFX A bir pción/S [^ch]ibir
|
||||
SFX A ibir epción/S cibir
|
||||
SFX A r ción/S hibir
|
||||
SFX A ecir ición/S [^ae]decir
|
||||
SFX A ecir icción/S [ae]decir
|
||||
SFX A r ción/S [^c]ir
|
||||
SFX A r ción/S [^u]cir
|
||||
SFX A ir ción/S ucir
|
||||
SFX B Y 3
|
||||
SFX B r dura/S [aií]r
|
||||
|
|
@ -202,14 +194,13 @@ SFX M o eza/S nto
|
|||
SFX M o eza/S [aie]sto
|
||||
SFX M o ez/S usto
|
||||
SFX M o uez/S go
|
||||
SFX N Y 19
|
||||
SFX N Y 18
|
||||
SFX N a illa/S [bdfhjlmnprstv]a
|
||||
SFX N o illo/S [bdhjlmnprstv]o
|
||||
SFX N 0 cilla/S ve
|
||||
SFX N za cilla/S za
|
||||
SFX N 0 ecilla/S d
|
||||
SFX N ión ioncilla/S [cglnstx]ión
|
||||
SFX N ón oncillo/S [^i]ón
|
||||
SFX N ón oncillo/S ón
|
||||
SFX N 0 cillo/S or
|
||||
SFX N zo cillo/S zo
|
||||
SFX N 0 cillo/S [djlnu]e
|
||||
|
|
@ -270,7 +261,7 @@ SFX T r ble/S ar
|
|||
SFX T er ible/S [^aeo]er
|
||||
SFX T er íble/S [aeo]er
|
||||
SFX T r ble/S ir
|
||||
SFX U Y 19
|
||||
SFX U Y 18
|
||||
SFX U a ita/S [bdfhjlmnprstv]a
|
||||
SFX U o ito/S [bdhjlmnprstv]o
|
||||
SFX U 0 cita/S ve
|
||||
|
|
@ -278,7 +269,6 @@ SFX U za cita/S za
|
|||
SFX U 0 ecita/S d
|
||||
SFX U ón oncito/S ón
|
||||
SFX U 0 cito/S or
|
||||
SFX U o íto/S eo
|
||||
SFX U zo cito/S zo
|
||||
SFX U 0 cito/S [djlnu]e
|
||||
SFX U z cecita/S z
|
||||
|
|
@ -290,7 +280,8 @@ SFX U co quito/S co
|
|||
SFX U go guito/S go
|
||||
SFX U ca quita/S ca
|
||||
SFX U ga guita/S ga
|
||||
SFX R Y 235
|
||||
SFX R Y 240
|
||||
SFX R ar ás ar
|
||||
SFX R r mos [aei]r
|
||||
SFX R ar áis ar
|
||||
SFX R r ba ar
|
||||
|
|
@ -346,10 +337,12 @@ SFX R 0 es ar
|
|||
SFX R ar áremos ar
|
||||
SFX R 0 eis ar
|
||||
SFX R 0 en ar
|
||||
SFX R ar á ar
|
||||
SFX R r d [aei]r
|
||||
SFX R r ndo ar
|
||||
SFX R ar ándose ar
|
||||
SFX R 0 se [aeií]r
|
||||
SFX R er és er
|
||||
SFX R er éis er
|
||||
SFX R er ía er
|
||||
SFX R er ías er
|
||||
|
|
@ -403,6 +396,7 @@ SFX R er yendo [aeo]er
|
|||
SFX R er yéndose [aeo]er
|
||||
SFX R r ndo ñer
|
||||
SFX R er éndose ñer
|
||||
SFX R er é er
|
||||
SFX R ir ís ir
|
||||
SFX R ir ía ir
|
||||
SFX R ir ías ir
|
||||
|
|
@ -526,6 +520,7 @@ SFX R r endo [gq]uir
|
|||
SFX R r éndose [gq]uir
|
||||
SFX R ir yendo [^gq]uir
|
||||
SFX R ir yéndose [^gq]uir
|
||||
SFX R ir í ir
|
||||
SFX E Y 73
|
||||
SFX E ar o ar
|
||||
SFX E r s [ae]r
|
||||
|
|
@ -600,9 +595,10 @@ SFX E cir zan cir
|
|||
SFX E gir jan gir
|
||||
SFX E uir an guir
|
||||
SFX E quir can quir
|
||||
SFX I Y 738
|
||||
SFX I Y 741
|
||||
SFX I ertar ierto ertar
|
||||
SFX I ertar iertas ertar
|
||||
SFX I ar ás ar
|
||||
SFX I ertar ierta ertar
|
||||
SFX I ertar iertan ertar
|
||||
SFX I ertar ierte ertar
|
||||
|
|
@ -1142,6 +1138,7 @@ SFX I usar úses usar
|
|||
SFX I usar úsen usar
|
||||
SFX I cer zco [ae]cer
|
||||
SFX I r s [ae]cer
|
||||
SFX I er és er
|
||||
SFX I r 0 [ae]cer
|
||||
SFX I r n [ae]cer
|
||||
SFX I cer zca [ae]cer
|
||||
|
|
@ -1277,6 +1274,7 @@ SFX I er igan oer
|
|||
SFX I er yan oer
|
||||
SFX I ir go sir
|
||||
SFX I ir es sir
|
||||
SFX I ir ís ir
|
||||
SFX I ir e sir
|
||||
SFX I ir en sir
|
||||
SFX I ir ga sir
|
||||
|
|
@ -1339,13 +1337,14 @@ SFX I ir yas uir
|
|||
SFX I ir yamos uir
|
||||
SFX I ir yáis uir
|
||||
SFX I ir yan uir
|
||||
SFX X Y 1137
|
||||
SFX X Y 1141
|
||||
SFX X er go oner
|
||||
SFX X r s oner
|
||||
SFX X r 0 oner
|
||||
SFX X r mos [ei]r
|
||||
SFX X er éis [^v]er
|
||||
SFX X er éis [es]ver
|
||||
SFX X er és [^v]er
|
||||
SFX X r n oner
|
||||
SFX X erer iero erer
|
||||
SFX X erer ieres erer
|
||||
|
|
@ -2018,26 +2017,26 @@ SFX X aber upierais aber
|
|||
SFX X aber upieseis aber
|
||||
SFX X aber upieran aber
|
||||
SFX X aber upiesen aber
|
||||
SFX X r ra ñer
|
||||
SFX X r se ñer
|
||||
SFX X r ras ñer
|
||||
SFX X r ses ñer
|
||||
SFX X er éramos ñer
|
||||
SFX X er ésemos ñer
|
||||
SFX X r rais ñer
|
||||
SFX X r seis ñer
|
||||
SFX X r ran ñer
|
||||
SFX X r sen ñer
|
||||
SFX X er iera [lv]er
|
||||
SFX X er iese [lv]er
|
||||
SFX X er ieras [lv]er
|
||||
SFX X er ieses [lv]er
|
||||
SFX X er iéramos [lv]er
|
||||
SFX X er iésemos [lv]er
|
||||
SFX X er ierais [lv]er
|
||||
SFX X er ieseis [lv]er
|
||||
SFX X er ieran [lv]er
|
||||
SFX X er iesen [lv]er
|
||||
SFX X r ra ñer
|
||||
SFX X r se ñer
|
||||
SFX X r ras ñer
|
||||
SFX X r ses ñer
|
||||
SFX X er éramos ñer
|
||||
SFX X er ésemos ñer
|
||||
SFX X r rais ñer
|
||||
SFX X r seis ñer
|
||||
SFX X r ran ñer
|
||||
SFX X r sen ñer
|
||||
SFX X er iera [lv]er
|
||||
SFX X er iese [lv]er
|
||||
SFX X er ieras [lv]er
|
||||
SFX X er ieses [lv]er
|
||||
SFX X er iéramos [lv]er
|
||||
SFX X er iésemos [lv]er
|
||||
SFX X er ierais [lv]er
|
||||
SFX X er ieseis [lv]er
|
||||
SFX X er ieran [lv]er
|
||||
SFX X er iesen [lv]er
|
||||
SFX X erir iriera erir
|
||||
SFX X erir iriese erir
|
||||
SFX X erir irieras erir
|
||||
|
|
@ -2298,16 +2297,16 @@ SFX X aber upieres aber
|
|||
SFX X aber upiéremos aber
|
||||
SFX X aber upiereis aber
|
||||
SFX X aber upieren aber
|
||||
SFX X r re ñer
|
||||
SFX X r res ñer
|
||||
SFX X er éremos ñer
|
||||
SFX X r reis ñer
|
||||
SFX X r ren ñer
|
||||
SFX X er iere [lv]er
|
||||
SFX X er ieres [lv]er
|
||||
SFX X er iéremos [lv]er
|
||||
SFX X er iereis [lv]er
|
||||
SFX X er ieren [lv]er
|
||||
SFX X r re ñer
|
||||
SFX X r res ñer
|
||||
SFX X er éremos ñer
|
||||
SFX X r reis ñer
|
||||
SFX X r ren ñer
|
||||
SFX X er iere [lv]er
|
||||
SFX X er ieres [lv]er
|
||||
SFX X er iéremos [lv]er
|
||||
SFX X er iereis [lv]er
|
||||
SFX X er ieren [lv]er
|
||||
SFX X erir iriere erir
|
||||
SFX X erir irieres erir
|
||||
SFX X erir iriéremos erir
|
||||
|
|
@ -2424,6 +2423,9 @@ SFX X ír yéremos oír
|
|||
SFX X ír yereis oír
|
||||
SFX X ír yeren oír
|
||||
SFX X r d [ei]r
|
||||
SFX X er é er
|
||||
SFX X ir í ir
|
||||
SFX X r 0 ír
|
||||
SFX X er iendo [^añ]er
|
||||
SFX X er iéndose [^añ]er
|
||||
SFX X er yendo aer
|
||||
|
|
@ -3063,6 +3065,10 @@ SFX Ì abar ábala abar
|
|||
SFX Ì abar ábalas abar
|
||||
SFX Ì abar ábalo abar
|
||||
SFX Ì abar ábalos abar
|
||||
SFX Ì r la r
|
||||
SFX Ì r las r
|
||||
SFX Ì r lo r
|
||||
SFX Ì r los r
|
||||
SFX Ì r dla r
|
||||
SFX Ì r dlas r
|
||||
SFX Ì r dlo r
|
||||
|
|
@ -3439,13 +3445,11 @@ SFX Ì utar útalas utar
|
|||
SFX Ì utar útala utar
|
||||
SFX Ì utar útalos utar
|
||||
SFX Ì utar útalo utar
|
||||
SFX Ì uzar úzalas uzar
|
||||
SFX Ì uzar úzala uzar
|
||||
SFX Ì uzar úzalos uzar
|
||||
SFX Ì uzar úzalo uzar
|
||||
SFX Í Y 172
|
||||
SFX Í Y 164
|
||||
SFX Í abar ábame abar
|
||||
SFX Í abar ábanos abar
|
||||
SFX Í r me r
|
||||
SFX Í r nos r
|
||||
SFX Í r dme r
|
||||
SFX Í r dnos r
|
||||
SFX Í ablar áblame ablar
|
||||
|
|
@ -3486,8 +3490,6 @@ SFX Í argar árgame argar
|
|||
SFX Í argar árganos argar
|
||||
SFX Í arrar árrame arrar
|
||||
SFX Í arrar árranos arrar
|
||||
SFX Í asar ásame asar
|
||||
SFX Í asar ásanos asar
|
||||
SFX Í atar átame atar
|
||||
SFX Í atar átanos atar
|
||||
SFX Í avar ávame avar
|
||||
|
|
@ -3502,8 +3504,6 @@ SFX Í eer éeme eer
|
|||
SFX Í eer éenos eer
|
||||
SFX Í egar égame egar
|
||||
SFX Í egar éganos egar
|
||||
SFX Í eger égeme eger
|
||||
SFX Í eger égenos eger
|
||||
SFX Í egrar égrame egrar
|
||||
SFX Í egrar égranos egrar
|
||||
SFX Í ejar éjame ejar
|
||||
|
|
@ -3528,8 +3528,6 @@ SFX Í ercar ércame ercar
|
|||
SFX Í ercar ércanos ercar
|
||||
SFX Í esar ésame esar
|
||||
SFX Í esar ésanos esar
|
||||
SFX Í escar éscame escar
|
||||
SFX Í escar éscanos escar
|
||||
SFX Í estar éstame estar
|
||||
SFX Í estar éstanos estar
|
||||
SFX Í etar étame etar
|
||||
|
|
@ -3540,8 +3538,6 @@ SFX Í evar évame evar
|
|||
SFX Í evar évanos evar
|
||||
SFX Í ibir íbeme ibir
|
||||
SFX Í ibir íbenos ibir
|
||||
SFX Í ibrar íbrame ibrar
|
||||
SFX Í ibrar íbranos ibrar
|
||||
SFX Í icar ícame icar
|
||||
SFX Í icar ícanos icar
|
||||
SFX Í iciar íciame iciar
|
||||
|
|
@ -3562,8 +3558,6 @@ SFX Í obrar óbrame obrar
|
|||
SFX Í obrar óbranos obrar
|
||||
SFX Í ocar ócame ocar
|
||||
SFX Í ocar ócanos ocar
|
||||
SFX Í ocer óceme ocer
|
||||
SFX Í ocer ócenos ocer
|
||||
SFX Í oger ógeme oger
|
||||
SFX Í oger ógenos oger
|
||||
SFX Í ojar ójame ojar
|
||||
|
|
@ -3619,6 +3613,8 @@ SFX Í uscar úscanos uscar
|
|||
SFX Î Y 86
|
||||
SFX Î abar ábale abar
|
||||
SFX Î abar ábales abar
|
||||
SFX Î r le r
|
||||
SFX Î r les r
|
||||
SFX Î r dle r
|
||||
SFX Î r dles r
|
||||
SFX Î ablar áblale ablar
|
||||
|
|
@ -3659,8 +3655,6 @@ SFX Î eter étele eter
|
|||
SFX Î eter ételes eter
|
||||
SFX Î evar évale evar
|
||||
SFX Î evar évales evar
|
||||
SFX Î ibir íbele ibir
|
||||
SFX Î ibir íbeles ibir
|
||||
SFX Î icar ícale icar
|
||||
SFX Î icar ícales icar
|
||||
SFX Î iciar íciale iciar
|
||||
|
|
@ -3703,7 +3697,7 @@ SFX Î untar úntales untar
|
|||
SFX Î untar úntale untar
|
||||
SFX Î uscar úscales uscar
|
||||
SFX Î uscar úscale uscar
|
||||
SFX Ï Y 288
|
||||
SFX Ï Y 300
|
||||
SFX Ï adir ádemela adir
|
||||
SFX Ï adir ádemelas adir
|
||||
SFX Ï adir ádemelo adir
|
||||
|
|
@ -3716,6 +3710,18 @@ SFX Ï adir ádesela adir
|
|||
SFX Ï adir ádeselas adir
|
||||
SFX Ï adir ádeselo adir
|
||||
SFX Ï adir ádeselos adir
|
||||
SFX Ï ir ímela ir
|
||||
SFX Ï ir ímelas ir
|
||||
SFX Ï ir ímelo ir
|
||||
SFX Ï ir ímelos ir
|
||||
SFX Ï ir ínosla ir
|
||||
SFX Ï ir ínoslas ir
|
||||
SFX Ï ir ínoslo ir
|
||||
SFX Ï ir ínoslos ir
|
||||
SFX Ï ir ísela ir
|
||||
SFX Ï ir íselas ir
|
||||
SFX Ï ir íselo ir
|
||||
SFX Ï ir íselos ir
|
||||
SFX Ï ir ídmela ir
|
||||
SFX Ï ir ídmelas ir
|
||||
SFX Ï ir ídmelo ir
|
||||
|
|
@ -3740,6 +3746,18 @@ SFX Ï ambiar ámbiasela ambiar
|
|||
SFX Ï ambiar ámbiaselas ambiar
|
||||
SFX Ï ambiar ámbiaselo ambiar
|
||||
SFX Ï ambiar ámbiaselos ambiar
|
||||
SFX Ï ar ámela ar
|
||||
SFX Ï ar ámelas ar
|
||||
SFX Ï ar ámelo ar
|
||||
SFX Ï ar ámelos ar
|
||||
SFX Ï ar ánosla ar
|
||||
SFX Ï ar ánoslas ar
|
||||
SFX Ï ar ánoslo ar
|
||||
SFX Ï ar ánoslos ar
|
||||
SFX Ï ar ásela ar
|
||||
SFX Ï ar áselas ar
|
||||
SFX Ï ar áselo ar
|
||||
SFX Ï ar áselos ar
|
||||
SFX Ï ar ádmela ar
|
||||
SFX Ï ar ádmelas ar
|
||||
SFX Ï ar ádmelo ar
|
||||
|
|
@ -3764,18 +3782,6 @@ SFX Ï ancar áncasela ancar
|
|||
SFX Ï ancar áncaselas ancar
|
||||
SFX Ï ancar áncaselo ancar
|
||||
SFX Ï ancar áncaselos ancar
|
||||
SFX Ï andar ándamela andar
|
||||
SFX Ï andar ándamelas andar
|
||||
SFX Ï andar ándamelo andar
|
||||
SFX Ï andar ándamelos andar
|
||||
SFX Ï andar ándanosla andar
|
||||
SFX Ï andar ándanoslas andar
|
||||
SFX Ï andar ándanoslo andar
|
||||
SFX Ï andar ándanoslos andar
|
||||
SFX Ï andar ándasela andar
|
||||
SFX Ï andar ándaselas andar
|
||||
SFX Ï andar ándaselo andar
|
||||
SFX Ï andar ándaselos andar
|
||||
SFX Ï antar ántamela antar
|
||||
SFX Ï antar ántamelas antar
|
||||
SFX Ï antar ántamelo antar
|
||||
|
|
@ -3800,18 +3806,6 @@ SFX Ï anzar ánzasela anzar
|
|||
SFX Ï anzar ánzaselas anzar
|
||||
SFX Ï anzar ánzaselo anzar
|
||||
SFX Ï anzar ánzaselos anzar
|
||||
SFX Ï ardar árdamela ardar
|
||||
SFX Ï ardar árdamelas ardar
|
||||
SFX Ï ardar árdamelo ardar
|
||||
SFX Ï ardar árdamelos ardar
|
||||
SFX Ï ardar árdanosla ardar
|
||||
SFX Ï ardar árdanoslas ardar
|
||||
SFX Ï ardar árdanoslo ardar
|
||||
SFX Ï ardar árdanoslos ardar
|
||||
SFX Ï ardar árdasela ardar
|
||||
SFX Ï ardar árdaselas ardar
|
||||
SFX Ï ardar árdaselo ardar
|
||||
SFX Ï ardar árdaselos ardar
|
||||
SFX Ï asar ásamela asar
|
||||
SFX Ï asar ásamelas asar
|
||||
SFX Ï asar ásamelo asar
|
||||
|
|
@ -3848,6 +3842,18 @@ SFX Ï eder édesela eder
|
|||
SFX Ï eder édeselas eder
|
||||
SFX Ï eder édeselo eder
|
||||
SFX Ï eder édeselos eder
|
||||
SFX Ï er émela er
|
||||
SFX Ï er émelas er
|
||||
SFX Ï er émelo er
|
||||
SFX Ï er émelos er
|
||||
SFX Ï er énosla er
|
||||
SFX Ï er énoslas er
|
||||
SFX Ï er énoslo er
|
||||
SFX Ï er énoslos er
|
||||
SFX Ï er ésela er
|
||||
SFX Ï er éselas er
|
||||
SFX Ï er éselo er
|
||||
SFX Ï er éselos er
|
||||
SFX Ï er édmela er
|
||||
SFX Ï er édmelas er
|
||||
SFX Ï er édmelo er
|
||||
|
|
@ -3992,11 +3998,15 @@ SFX Ï urar úraselas urar
|
|||
SFX Ï urar úrasela urar
|
||||
SFX Ï urar úraselos urar
|
||||
SFX Ï urar úraselo urar
|
||||
SFX Ð Y 56
|
||||
SFX Ð Y 60
|
||||
SFX Ð egar iégala egar
|
||||
SFX Ð egar iégalas egar
|
||||
SFX Ð egar iégalo egar
|
||||
SFX Ð egar iégalos egar
|
||||
SFX Ð r la r
|
||||
SFX Ð r las r
|
||||
SFX Ð r lo r
|
||||
SFX Ð r los r
|
||||
SFX Ð r dla r
|
||||
SFX Ð r dlas r
|
||||
SFX Ð r dlo r
|
||||
|
|
@ -4049,9 +4059,11 @@ SFX Ð ostrar uéstrala ostrar
|
|||
SFX Ð ostrar uéstralas ostrar
|
||||
SFX Ð ostrar uéstralo ostrar
|
||||
SFX Ð ostrar uéstralos ostrar
|
||||
SFX Ñ Y 24
|
||||
SFX Ñ Y 26
|
||||
SFX Ñ ecer éceme ecer
|
||||
SFX Ñ ecer écenos ecer
|
||||
SFX Ñ r me r
|
||||
SFX Ñ r nos r
|
||||
SFX Ñ r dme r
|
||||
SFX Ñ r dnos r
|
||||
SFX Ñ ender iéndeme ender
|
||||
|
|
@ -4074,9 +4086,11 @@ SFX Ñ order uérdeme order
|
|||
SFX Ñ order uérdenos order
|
||||
SFX Ñ ostrar uéstrame ostrar
|
||||
SFX Ñ ostrar uéstranos ostrar
|
||||
SFX Ò Y 20
|
||||
SFX Ò Y 22
|
||||
SFX Ò ecer écele ecer
|
||||
SFX Ò ecer éceles ecer
|
||||
SFX Ò r le r
|
||||
SFX Ò r les r
|
||||
SFX Ò r dle r
|
||||
SFX Ò r dles r
|
||||
SFX Ò egar iégale egar
|
||||
|
|
@ -4095,7 +4109,7 @@ SFX Ò order uérdele order
|
|||
SFX Ò order uérdeles order
|
||||
SFX Ò ostrar uéstrale ostrar
|
||||
SFX Ò ostrar uéstrales ostrar
|
||||
SFX Ó Y 96
|
||||
SFX Ó Y 120
|
||||
SFX Ó ecer écemela ecer
|
||||
SFX Ó ecer écemelas ecer
|
||||
SFX Ó ecer écemelo ecer
|
||||
|
|
@ -4108,6 +4122,18 @@ SFX Ó ecer écesela ecer
|
|||
SFX Ó ecer éceselas ecer
|
||||
SFX Ó ecer éceselo ecer
|
||||
SFX Ó ecer éceselos ecer
|
||||
SFX Ó er émela er
|
||||
SFX Ó er émelas er
|
||||
SFX Ó er émelo er
|
||||
SFX Ó er émelos er
|
||||
SFX Ó er énosla er
|
||||
SFX Ó er énoslas er
|
||||
SFX Ó er énoslo er
|
||||
SFX Ó er énoslos er
|
||||
SFX Ó er ésela er
|
||||
SFX Ó er éselas er
|
||||
SFX Ó er éselo er
|
||||
SFX Ó er éselos er
|
||||
SFX Ó er édmela er
|
||||
SFX Ó er édmelas er
|
||||
SFX Ó er édmelo er
|
||||
|
|
@ -4132,6 +4158,18 @@ SFX Ó iar íasela iar
|
|||
SFX Ó iar íaselas iar
|
||||
SFX Ó iar íaselo iar
|
||||
SFX Ó iar íaselos iar
|
||||
SFX Ó ar ámela ar
|
||||
SFX Ó ar ámelas ar
|
||||
SFX Ó ar ámelo ar
|
||||
SFX Ó ar ámelos ar
|
||||
SFX Ó ar ánosla ar
|
||||
SFX Ó ar ánoslas ar
|
||||
SFX Ó ar ánoslo ar
|
||||
SFX Ó ar ánoslos ar
|
||||
SFX Ó ar ásela ar
|
||||
SFX Ó ar áselas ar
|
||||
SFX Ó ar áselo ar
|
||||
SFX Ó ar áselos ar
|
||||
SFX Ó ar ádmela ar
|
||||
SFX Ó ar ádmelas ar
|
||||
SFX Ó ar ádmelo ar
|
||||
|
|
@ -4197,6 +4235,10 @@ SFX Ô aer áela aer
|
|||
SFX Ô aer áelas aer
|
||||
SFX Ô aer áelo aer
|
||||
SFX Ô aer áelos aer
|
||||
SFX Ô r la r
|
||||
SFX Ô r las r
|
||||
SFX Ô r lo r
|
||||
SFX Ô r los r
|
||||
SFX Ô r dla r
|
||||
SFX Ô r dlas r
|
||||
SFX Ô r dlo r
|
||||
|
|
@ -4213,10 +4255,6 @@ SFX Ô edir ídela edir
|
|||
SFX Ô edir ídelas edir
|
||||
SFX Ô edir ídelo edir
|
||||
SFX Ô edir ídelos edir
|
||||
SFX Ô egir ígela egir
|
||||
SFX Ô egir ígelas egir
|
||||
SFX Ô egir ígelo egir
|
||||
SFX Ô egir ígelos egir
|
||||
SFX Ô eguir íguela eguir
|
||||
SFX Ô eguir íguelas eguir
|
||||
SFX Ô eguir íguelo eguir
|
||||
|
|
@ -4253,9 +4291,11 @@ SFX Ô uir úyelas uir
|
|||
SFX Ô uir úyela uir
|
||||
SFX Ô uir úyelos uir
|
||||
SFX Ô uir úyelo uir
|
||||
SFX Õ Y 26
|
||||
SFX Õ Y 28
|
||||
SFX Õ aber ábeme aber
|
||||
SFX Õ aber ábenos aber
|
||||
SFX Õ r me r
|
||||
SFX Õ r nos r
|
||||
SFX Õ r dme r
|
||||
SFX Õ r dnos r
|
||||
SFX Õ aer áeme aer
|
||||
|
|
@ -4280,9 +4320,11 @@ SFX Õ r me ver
|
|||
SFX Õ r nos ver
|
||||
SFX Õ ucir úceme ucir
|
||||
SFX Õ ucir úcenos ucir
|
||||
SFX Ö Y 14
|
||||
SFX Ö Y 16
|
||||
SFX Ö aber ábele aber
|
||||
SFX Ö aber ábeles aber
|
||||
SFX Ö r le r
|
||||
SFX Ö r les r
|
||||
SFX Ö r dle r
|
||||
SFX Ö r dles r
|
||||
SFX Ö aer áele aer
|
||||
|
|
@ -4295,7 +4337,7 @@ SFX Ö er le ner
|
|||
SFX Ö er les ner
|
||||
SFX Ö r le ver
|
||||
SFX Ö r les ver
|
||||
SFX Ø Y 60
|
||||
SFX Ø Y 84
|
||||
SFX Ø aer áemela aer
|
||||
SFX Ø aer áemelas aer
|
||||
SFX Ø aer áemelo aer
|
||||
|
|
@ -4308,6 +4350,18 @@ SFX Ø aer áesela aer
|
|||
SFX Ø aer áeselas aer
|
||||
SFX Ø aer áeselo aer
|
||||
SFX Ø aer áeselos aer
|
||||
SFX Ø er émela er
|
||||
SFX Ø er émelas er
|
||||
SFX Ø er émelo er
|
||||
SFX Ø er émelos er
|
||||
SFX Ø er énosla er
|
||||
SFX Ø er énoslas er
|
||||
SFX Ø er énoslo er
|
||||
SFX Ø er énoslos er
|
||||
SFX Ø er ésela er
|
||||
SFX Ø er éselas er
|
||||
SFX Ø er éselo er
|
||||
SFX Ø er éselos er
|
||||
SFX Ø er édmela er
|
||||
SFX Ø er édmelas er
|
||||
SFX Ø er édmelo er
|
||||
|
|
@ -4332,6 +4386,18 @@ SFX Ø edir ídesela edir
|
|||
SFX Ø edir ídeselas edir
|
||||
SFX Ø edir ídeselo edir
|
||||
SFX Ø edir ídeselos edir
|
||||
SFX Ø ir ímela ir
|
||||
SFX Ø ir ímelas ir
|
||||
SFX Ø ir ímelo ir
|
||||
SFX Ø ir ímelos ir
|
||||
SFX Ø ir ínosla ir
|
||||
SFX Ø ir ínoslas ir
|
||||
SFX Ø ir ínoslo ir
|
||||
SFX Ø ir ínoslos ir
|
||||
SFX Ø ir ísela ir
|
||||
SFX Ø ir íselas ir
|
||||
SFX Ø ir íselo ir
|
||||
SFX Ø ir íselos ir
|
||||
SFX Ø ir ídmela ir
|
||||
SFX Ø ir ídmelas ir
|
||||
SFX Ø ir ídmelo ir
|
||||
|
|
@ -6219,8 +6285,9 @@ SFX ó r éndome ucir
|
|||
SFX ó r éndonos ucir
|
||||
SFX ó r éndoos ucir
|
||||
SFX ó r éndote ucir
|
||||
SFX ô Y 137
|
||||
SFX ô Y 136
|
||||
SFX ô abar ábate abar
|
||||
SFX ô r te [aei]r
|
||||
SFX ô r os r
|
||||
SFX ô acar ácate acar
|
||||
SFX ô achar áchate achar
|
||||
|
|
@ -6264,7 +6331,6 @@ SFX ô azar ázate azar
|
|||
SFX ô ear éate ear
|
||||
SFX ô ecar écate ecar
|
||||
SFX ô echar échate echar
|
||||
SFX ô ectar éctate ectar
|
||||
SFX ô edar édate edar
|
||||
SFX ô egar égate egar
|
||||
SFX ô eger égete eger
|
||||
|
|
@ -6307,7 +6373,6 @@ SFX ô intar íntate intar
|
|||
SFX ô irar írate irar
|
||||
SFX ô ir íos ir
|
||||
SFX ô istar ístate istar
|
||||
SFX ô istrar ístrate istrar
|
||||
SFX ô itar ítate itar
|
||||
SFX ô itir ítete itir
|
||||
SFX ô izar ízate izar
|
||||
|
|
@ -6357,8 +6422,9 @@ SFX ô urlar úrlate urlar
|
|||
SFX ô uscar úscate uscar
|
||||
SFX ô ustar ústate ustar
|
||||
SFX ô ustrar ústrate ustrar
|
||||
SFX õ Y 30
|
||||
SFX õ Y 31
|
||||
SFX õ aer áete aer
|
||||
SFX õ r te [aei]r
|
||||
SFX õ r os [^i]r
|
||||
SFX õ ecer écete ecer
|
||||
SFX õ egar iégate egar
|
||||
|
|
@ -6388,8 +6454,9 @@ SFX õ over uévete over
|
|||
SFX õ uar úate uar
|
||||
SFX õ unir únete unir
|
||||
SFX õ ir íos ir
|
||||
SFX ö Y 17
|
||||
SFX ö Y 18
|
||||
SFX ö aer áete aer
|
||||
SFX ö r te [eií]r
|
||||
SFX ö r os [^i]r
|
||||
SFX ö aler álete aler
|
||||
SFX ö cer zte cer
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -875,6 +875,7 @@ application/x-ustar ustar
|
|||
application/x-wais-source src
|
||||
application/x-wingz wz
|
||||
application/x-x509-ca-cert crt der
|
||||
application/x-xcf xcf
|
||||
application/x-xfig fig
|
||||
application/x-xpinstall xpi
|
||||
application/x400-bp
|
||||
|
|
@ -1145,7 +1146,6 @@ image/x-portable-graymap pgm
|
|||
image/x-portable-pixmap ppm
|
||||
image/x-rgb rgb
|
||||
image/x-xbitmap xbm
|
||||
image/x-xcf xcf
|
||||
image/x-xpixmap xpm
|
||||
image/x-xwindowdump xwd
|
||||
message/cpim
|
||||
|
|
|
|||
|
|
@ -35,8 +35,6 @@
|
|||
// is U+2010 but that does not render with the default Times font on macOS as of Monterey
|
||||
// and Qt 15.5 See https://bugs.launchpad.net/bugs/1951467 and can be easily reproduced
|
||||
// by converting a plain text file with the --pdf-hyphenate option
|
||||
// https://bugs.chromium.org/p/chromium/issues/detail?id=1267606 (fix released Feb 1 2022 v98)
|
||||
// See also settings.pyj
|
||||
if (HYPHEN_CHAR) {
|
||||
for (const elem of document.getElementsByTagName('*')) {
|
||||
if (elem.style) {
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
|
|
@ -91,7 +92,7 @@ def build_cache_dir():
|
|||
_cache_dir_built = True
|
||||
try:
|
||||
os.mkdir(ans)
|
||||
except OSError as err:
|
||||
except EnvironmentError as err:
|
||||
if err.errno != errno.EEXIST:
|
||||
raise
|
||||
return ans
|
||||
|
|
@ -99,7 +100,7 @@ def build_cache_dir():
|
|||
|
||||
def require_git_master(branch='master'):
|
||||
if subprocess.check_output(['git', 'symbolic-ref', '--short', 'HEAD']).decode('utf-8').strip() != branch:
|
||||
raise SystemExit(f'You must be in the {branch} git branch')
|
||||
raise SystemExit('You must be in the {} git branch'.format(branch))
|
||||
|
||||
|
||||
def require_clean_git():
|
||||
|
|
@ -224,7 +225,7 @@ def run_cmd(self, cmd, opts):
|
|||
st = time.time()
|
||||
self.running(cmd)
|
||||
cmd.run(opts)
|
||||
self.info(f'* {command_names[cmd]} took {time.time() - st:.1f} seconds')
|
||||
self.info('* %s took %.1f seconds' % (command_names[cmd], time.time() - st))
|
||||
if os.environ.get('CI'):
|
||||
self.info('::endgroup::')
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
# License: GPLv3 Copyright: 2017, Kovid Goyal <kovid at kovidgoyal.net>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,11 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
__copyright__ = '2009, Kovid Goyal <kovid@kovidgoyal.net>'
|
||||
__docformat__ = 'restructuredtext en'
|
||||
|
||||
import textwrap, os, shlex, subprocess, glob, shutil, sys, json, errno, sysconfig
|
||||
import textwrap, os, shlex, subprocess, glob, shutil, sys, json, errno
|
||||
from collections import namedtuple
|
||||
|
||||
from setup import Command, islinux, isbsd, isfreebsd, ismacos, ishaiku, SRC, iswindows
|
||||
|
|
@ -27,7 +28,7 @@ def init_symbol_name(name):
|
|||
|
||||
|
||||
def absolutize(paths):
|
||||
return list({x if os.path.isabs(x) else os.path.join(SRC, x.replace('/', os.sep)) for x in paths})
|
||||
return list(set([x if os.path.isabs(x) else os.path.join(SRC, x.replace('/', os.sep)) for x in paths]))
|
||||
|
||||
|
||||
class Extension:
|
||||
|
|
@ -166,20 +167,9 @@ def read_extensions():
|
|||
return ans
|
||||
|
||||
|
||||
def get_python_include_paths():
|
||||
ans = []
|
||||
for name in sysconfig.get_path_names():
|
||||
if 'include' in name:
|
||||
ans.append(name)
|
||||
|
||||
def gp(x):
|
||||
return sysconfig.get_path(x)
|
||||
|
||||
return sorted(frozenset(filter(None, map(gp, sorted(ans)))))
|
||||
|
||||
|
||||
def init_env(debug=False, sanitize=False):
|
||||
from setup.build_environment import win_ld, is64bit, win_inc, win_lib, NMAKE, win_cc
|
||||
from distutils import sysconfig
|
||||
linker = None
|
||||
if isunix:
|
||||
cc = os.environ.get('CC', 'gcc')
|
||||
|
|
@ -212,20 +202,17 @@ def init_env(debug=False, sanitize=False):
|
|||
ldflags.append('-shared')
|
||||
|
||||
if islinux or isbsd or ishaiku:
|
||||
cflags.extend('-I' + x for x in get_python_include_paths())
|
||||
ldlib = sysconfig.get_config_var('LIBDIR')
|
||||
if ldlib:
|
||||
ldflags += ['-L' + ldlib]
|
||||
ldlib = sysconfig.get_config_var('VERSION')
|
||||
if ldlib:
|
||||
ldflags += ['-lpython' + ldlib + sys.abiflags]
|
||||
ldflags += (sysconfig.get_config_var('LINKFORSHARED') or '').split()
|
||||
cflags.append('-I'+sysconfig.get_python_inc())
|
||||
# getattr(..., 'abiflags') is for PY2 compat, since PY2 has no abiflags
|
||||
# member
|
||||
ldflags.append('-lpython{}{}'.format(
|
||||
sysconfig.get_config_var('VERSION'), getattr(sys, 'abiflags', '')))
|
||||
|
||||
if ismacos:
|
||||
cflags.append('-D_OSX')
|
||||
ldflags.extend('-bundle -undefined dynamic_lookup'.split())
|
||||
cflags.extend(['-fno-common', '-dynamic'])
|
||||
cflags.extend('-I' + x for x in get_python_include_paths())
|
||||
cflags.append('-I'+sysconfig.get_python_inc())
|
||||
|
||||
if iswindows:
|
||||
cc = cxx = win_cc
|
||||
|
|
@ -246,8 +233,8 @@ def init_env(debug=False, sanitize=False):
|
|||
for p in win_lib:
|
||||
if p:
|
||||
ldflags.append('/LIBPATH:'+p)
|
||||
cflags.extend('-I' + x for x in get_python_include_paths())
|
||||
ldflags.append('/LIBPATH:'+os.path.join(sysconfig.get_config_var('prefix'), 'libs'))
|
||||
cflags.append('-I%s'%sysconfig.get_python_inc())
|
||||
ldflags.append('/LIBPATH:'+os.path.join(sysconfig.PREFIX, 'libs'))
|
||||
linker = win_ld
|
||||
return namedtuple('Environment', 'cc cxx cflags ldflags linker make')(
|
||||
cc=cc, cxx=cxx, cflags=cflags, ldflags=ldflags, linker=linker, make=NMAKE if iswindows else 'make')
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
# License: GPL v3 Copyright: 2020, Kovid Goyal <kovid at kovidgoyal.net>
|
||||
|
||||
from datetime import date
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
|
|
@ -15,19 +16,7 @@ def __init__(self, filename, lineno, msg):
|
|||
self.filename, self.lineno, self.msg = filename, lineno, msg
|
||||
|
||||
def __str__(self):
|
||||
return f'{self.filename}:{self.lineno}: {self.msg}'
|
||||
|
||||
|
||||
def checkable_python_files(SRC):
|
||||
for dname in ('odf', 'calibre'):
|
||||
for x in os.walk(os.path.join(SRC, dname)):
|
||||
for f in x[-1]:
|
||||
y = os.path.join(x[0], f)
|
||||
if (f.endswith('.py') and f not in (
|
||||
'dict_data.py', 'unicodepoints.py', 'krcodepoints.py',
|
||||
'jacodepoints.py', 'vncodepoints.py', 'zhcodepoints.py') and
|
||||
'prs500/driver.py' not in y) and not f.endswith('_ui.py'):
|
||||
yield y
|
||||
return '%s:%s: %s' % (self.filename, self.lineno, self.msg)
|
||||
|
||||
|
||||
class Check(Command):
|
||||
|
|
@ -37,7 +26,15 @@ class Check(Command):
|
|||
CACHE = 'check.json'
|
||||
|
||||
def get_files(self):
|
||||
yield from checkable_python_files(self.SRC)
|
||||
for dname in ('odf', 'calibre'):
|
||||
for x in os.walk(self.j(self.SRC, dname)):
|
||||
for f in x[-1]:
|
||||
y = self.j(x[0], f)
|
||||
if (f.endswith('.py') and f not in (
|
||||
'dict_data.py', 'unicodepoints.py', 'krcodepoints.py',
|
||||
'jacodepoints.py', 'vncodepoints.py', 'zhcodepoints.py') and
|
||||
'prs500/driver.py' not in y) and not f.endswith('_ui.py'):
|
||||
yield y
|
||||
|
||||
for x in os.walk(self.j(self.d(self.SRC), 'recipes')):
|
||||
for f in x[-1]:
|
||||
|
|
@ -94,7 +91,7 @@ def run(self, opts):
|
|||
try:
|
||||
with open(self.cache_file, 'rb') as f:
|
||||
cache = json.load(f)
|
||||
except OSError as err:
|
||||
except EnvironmentError as err:
|
||||
if err.errno != errno.ENOENT:
|
||||
raise
|
||||
dirty_files = tuple(f for f in self.get_files() if not self.is_cache_valid(f, cache))
|
||||
|
|
@ -117,24 +114,6 @@ def report_errors(self, errors):
|
|||
def clean(self):
|
||||
try:
|
||||
os.remove(self.cache_file)
|
||||
except OSError as err:
|
||||
except EnvironmentError as err:
|
||||
if err.errno != errno.ENOENT:
|
||||
raise
|
||||
|
||||
|
||||
class UpgradeSourceCode(Command):
|
||||
|
||||
description = 'Upgrade python source code'
|
||||
|
||||
def run(self, opts):
|
||||
files = []
|
||||
for f in os.listdir(os.path.dirname(os.path.abspath(__file__))):
|
||||
q = os.path.join('setup', f)
|
||||
if f.endswith('.py') and f not in ('linux-installer.py',) and not os.path.isdir(q):
|
||||
files.append(q)
|
||||
for path in checkable_python_files(self.SRC):
|
||||
q = path.replace(os.sep, '/')
|
||||
if '/metadata/sources/' in q or '/store/stores/' in q:
|
||||
continue
|
||||
files.append(q)
|
||||
subprocess.call(['pyupgrade', '--py37-plus'] + files)
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
|
|
@ -12,7 +13,7 @@
|
|||
'git_version',
|
||||
'develop', 'install',
|
||||
'kakasi', 'rapydscript', 'cacerts', 'recent_uas', 'resources',
|
||||
'check', 'test', 'test_rs', 'upgrade_source_code',
|
||||
'check', 'test', 'test_rs',
|
||||
'sdist', 'bootstrap', 'extdev',
|
||||
'manual', 'tag_release',
|
||||
'upload_to_server',
|
||||
|
|
@ -65,9 +66,8 @@
|
|||
from setup.gui import GUI
|
||||
gui = GUI()
|
||||
|
||||
from setup.check import Check, UpgradeSourceCode
|
||||
from setup.check import Check
|
||||
check = Check()
|
||||
upgrade_source_code = UpgradeSourceCode()
|
||||
|
||||
from setup.test import Test, TestRS
|
||||
test = Test()
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
# License: GPL v3 Copyright: 2020, Kovid Goyal <kovid at kovidgoyal.net>
|
||||
|
||||
import os
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
__copyright__ = '2013, Kovid Goyal <kovid at kovidgoyal.net>'
|
||||
|
|
@ -9,7 +10,7 @@
|
|||
|
||||
|
||||
def get_branch_name(rev):
|
||||
return subprocess.check_output(['git', 'name-rev', '--name-only', '--refs=refs/heads/*', rev]).decode('utf-8').strip()
|
||||
return subprocess.check_output(['git', 'name-rev', '--name-only', rev]).decode('utf-8').strip()
|
||||
|
||||
|
||||
base = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
__copyright__ = '2014, Kovid Goyal <kovid at kovidgoyal.net>'
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
# License: GPLv3 Copyright: 2008, Kovid Goyal <kovid at kovidgoyal.net>
|
||||
|
||||
|
||||
|
|
@ -54,7 +55,7 @@ def __call__(self, match):
|
|||
print('Working on bug:', summary)
|
||||
if int(bug) > 100000 and action != 'See':
|
||||
self.close_bug(bug, action)
|
||||
return match.group() + f' [{summary}]({LAUNCHPAD_BUG % bug})'
|
||||
return match.group() + ' [%s](%s)' % (summary, LAUNCHPAD_BUG % bug)
|
||||
return match.group() + ' (%s)' % summary
|
||||
return match.group()
|
||||
|
||||
|
|
@ -65,7 +66,7 @@ def close_bug(self, bug, action):
|
|||
'calibre is usually released every alternate Friday.'
|
||||
)
|
||||
action += 'ed'
|
||||
msg = '{} in branch {}. {}'.format(action, 'master', suffix)
|
||||
msg = '%s in branch %s. %s' % (action, 'master', suffix)
|
||||
msg = msg.replace('Fixesed', 'Fixed')
|
||||
msg += '\n\n status fixreleased'
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
# License: GPLv3 Copyright: 2019, Eli Schwartz <eschwartz@archlinux.org>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
__copyright__ = '2011, Kovid Goyal <kovid@kovidgoyal.net>'
|
||||
|
|
@ -149,7 +150,7 @@ def __call__(self):
|
|||
existing_assets = self.existing_assets(release['id'])
|
||||
for path, desc in self.files.items():
|
||||
self.info('')
|
||||
url = self.API + 'repos/{}/{}/releases/assets/{{}}'.format(
|
||||
url = self.API + 'repos/%s/%s/releases/assets/{}' % (
|
||||
self.username, self.reponame
|
||||
)
|
||||
fname = os.path.basename(path)
|
||||
|
|
@ -205,7 +206,7 @@ def clean_older_releases(self, releases):
|
|||
|
||||
def do_upload(self, url, path, desc, fname):
|
||||
mime_type = mimetypes.guess_type(fname)[0] or 'application/octet-stream'
|
||||
self.info(f'Uploading to GitHub: {fname} ({mime_type})')
|
||||
self.info('Uploading to GitHub: %s (%s)' % (fname, mime_type))
|
||||
with ReadFileWithProgressReporting(path) as f:
|
||||
return self.requests.post(
|
||||
url,
|
||||
|
|
@ -228,7 +229,7 @@ def already_exists(self, r):
|
|||
return error_code == 'already_exists'
|
||||
|
||||
def existing_assets(self, release_id):
|
||||
url = self.API + 'repos/{}/{}/releases/{}/assets'.format(
|
||||
url = self.API + 'repos/%s/%s/releases/%s/assets' % (
|
||||
self.username, self.reponame, release_id
|
||||
)
|
||||
r = self.requests.get(url)
|
||||
|
|
@ -237,7 +238,7 @@ def existing_assets(self, release_id):
|
|||
return {asset['name']: asset['id'] for asset in r.json()}
|
||||
|
||||
def releases(self):
|
||||
url = self.API + f'repos/{self.username}/{self.reponame}/releases'
|
||||
url = self.API + 'repos/%s/%s/releases' % (self.username, self.reponame)
|
||||
r = self.requests.get(url)
|
||||
if r.status_code != 200:
|
||||
self.fail(r, 'Failed to list releases')
|
||||
|
|
@ -249,7 +250,7 @@ def create_release(self, releases):
|
|||
# Check for existing release
|
||||
if release['tag_name'] == self.current_tag_name:
|
||||
return release
|
||||
url = self.API + f'repos/{self.username}/{self.reponame}/releases'
|
||||
url = self.API + 'repos/%s/%s/releases' % (self.username, self.reponame)
|
||||
r = self.requests.post(
|
||||
url,
|
||||
data=json.dumps({
|
||||
|
|
@ -274,7 +275,7 @@ def generate_index(): # {{{
|
|||
releases = set()
|
||||
for x in os.listdir('.'):
|
||||
if os.path.isdir(x) and '.' in x:
|
||||
releases.add(tuple(int(y) for y in x.split('.')))
|
||||
releases.add(tuple((int(y) for y in x.split('.'))))
|
||||
rmap = OrderedDict()
|
||||
for rnum in sorted(releases, reverse=True):
|
||||
series = rnum[:2] if rnum[0] == 0 else rnum[:1]
|
||||
|
|
@ -300,10 +301,10 @@ def generate_index(): # {{{
|
|||
body.append(
|
||||
'<li><a href="{0}.html" title="Releases in the {0}.x series">{0}.x</a>\xa0\xa0\xa0<span style="font-size:smaller">[{1} releases]</span></li>'
|
||||
.format( # noqa
|
||||
'.'.join(map(str, series)), len(rmap[series])
|
||||
'.'.join(map(type(''), series)), len(rmap[series])
|
||||
)
|
||||
)
|
||||
body = '<ul>{}</ul>'.format(' '.join(body))
|
||||
body = '<ul>{0}</ul>'.format(' '.join(body))
|
||||
index = template.format(
|
||||
title='Previous calibre releases',
|
||||
style=style,
|
||||
|
|
@ -314,13 +315,13 @@ def generate_index(): # {{{
|
|||
f.write(index.encode('utf-8'))
|
||||
|
||||
for series, releases in rmap.items():
|
||||
sname = '.'.join(map(str, series))
|
||||
sname = '.'.join(map(type(''), series))
|
||||
body = [
|
||||
'<li><a href="{0}/" title="Release {0}">{0}</a></li>'.format(
|
||||
'.'.join(map(str, r))
|
||||
'.'.join(map(type(''), r))
|
||||
) for r in releases
|
||||
]
|
||||
body = '<ul class="release-list">{}</ul>'.format(' '.join(body))
|
||||
body = '<ul class="release-list">{0}</ul>'.format(' '.join(body))
|
||||
index = template.format(
|
||||
title='Previous calibre releases (%s.x)' % sname,
|
||||
style=style,
|
||||
|
|
@ -331,7 +332,7 @@ def generate_index(): # {{{
|
|||
f.write(index.encode('utf-8'))
|
||||
|
||||
for r in releases:
|
||||
rname = '.'.join(map(str, r))
|
||||
rname = '.'.join(map(type(''), r))
|
||||
os.chdir(rname)
|
||||
try:
|
||||
body = []
|
||||
|
|
@ -345,7 +346,7 @@ def generate_index(): # {{{
|
|||
) for x in windows
|
||||
]
|
||||
body.append(
|
||||
'<dt>Windows</dt><dd><ul>{}</ul></dd>'.format(
|
||||
'<dt>Windows</dt><dd><ul>{0}</ul></dd>'.format(
|
||||
' '.join(windows)
|
||||
)
|
||||
)
|
||||
|
|
@ -372,7 +373,7 @@ def generate_index(): # {{{
|
|||
) for x in linux
|
||||
]
|
||||
body.append(
|
||||
'<dt>Linux</dt><dd><ul>{}</ul></dd>'.format(
|
||||
'<dt>Linux</dt><dd><ul>{0}</ul></dd>'.format(
|
||||
' '.join(linux)
|
||||
)
|
||||
)
|
||||
|
|
@ -383,7 +384,7 @@ def generate_index(): # {{{
|
|||
.format(source[0], 'Source code (all platforms)')
|
||||
)
|
||||
|
||||
body = '<dl>{}</dl>'.format(''.join(body))
|
||||
body = '<dl>{0}</dl>'.format(''.join(body))
|
||||
index = template.format(
|
||||
title='calibre release (%s)' % rname,
|
||||
style=style,
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:fdm=marker:ai
|
||||
# License: GPLv3 Copyright: 2019, Kovid Goyal <kovid at kovidgoyal.net>
|
||||
from __future__ import absolute_import, division, print_function, unicode_literals
|
||||
|
||||
import glob
|
||||
import hashlib
|
||||
|
|
@ -98,7 +100,7 @@ def run(self, opts):
|
|||
for dic in dics:
|
||||
with open(os.path.join(output_dir, dic), 'rb') as f:
|
||||
m.update(f.read())
|
||||
hsh = str(m.hexdigest())
|
||||
hsh = type('')(m.hexdigest())
|
||||
buf = BytesIO()
|
||||
with tarfile.TarFile(fileobj=buf, mode='w') as tf:
|
||||
for dic in dics:
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
|
||||
# License: GPLv3 Copyright: 2009, Kovid Goyal <kovid at kovidgoyal.net>
|
||||
|
||||
|
||||
|
|
@ -154,15 +155,12 @@ def run(self, opts):
|
|||
self.success()
|
||||
|
||||
def install_env_module(self):
|
||||
import sysconfig
|
||||
libdir = os.path.join(
|
||||
self.opts.staging_root, sysconfig.get_config_var('PLATLIBDIR') or 'lib',
|
||||
os.path.basename(sysconfig.get_config_var('DESTLIB') or sysconfig.get_config_var('LIBDEST') or f'python{sysconfig.get_python_version()}'),
|
||||
'site-packages')
|
||||
import distutils.sysconfig as s
|
||||
libdir = s.get_python_lib(prefix=self.opts.staging_root)
|
||||
try:
|
||||
if not os.path.exists(libdir):
|
||||
os.makedirs(libdir)
|
||||
except OSError:
|
||||
except EnvironmentError:
|
||||
self.warn('Cannot install calibre environment module to: '+libdir)
|
||||
else:
|
||||
path = os.path.join(libdir, 'init_calibre.py')
|
||||
|
|
@ -366,7 +364,7 @@ def add_options(self, parser):
|
|||
def pre_sub_commands(self, opts):
|
||||
tdir = self.j(self.d(self.SRC), 'translations')
|
||||
clone_cmd = [
|
||||
'git', 'clone', f'https://github.com/{self.TRANSLATIONS_REPO}.git', 'translations']
|
||||
'git', 'clone', 'https://github.com/{}.git'.format(self.TRANSLATIONS_REPO), 'translations']
|
||||
if opts.ephemeral:
|
||||
if os.path.exists(tdir):
|
||||
shutil.rmtree(tdir)
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
# License: GPLv3 Copyright: 2016, Kovid Goyal <kovid at kovidgoyal.net>
|
||||
|
||||
|
||||
|
|
@ -12,9 +13,7 @@
|
|||
|
||||
def get_paths():
|
||||
base = d(d(os.path.abspath(__file__)))
|
||||
traditional_bypy_location = os.path.join(d(base), 'bypy')
|
||||
compat_bypy_location = os.path.join(base, 'bypy', 'b', 'bypy-old')
|
||||
bypy = compat_bypy_location if os.path.exists(compat_bypy_location) else traditional_bypy_location
|
||||
bypy = os.path.join(d(base), 'bypy')
|
||||
bypy = os.environ.get('BYPY_LOCATION', bypy)
|
||||
if not os.path.isdir(bypy):
|
||||
raise SystemExit(
|
||||
|
|
@ -85,7 +84,7 @@ def build_single(which='windows', bitness='64', shutdown=True, sign_installers=T
|
|||
dest = os.path.join(base, 'dist', x)
|
||||
try:
|
||||
os.remove(dest)
|
||||
except OSError:
|
||||
except EnvironmentError:
|
||||
pass
|
||||
os.link(src, dest)
|
||||
if shutdown:
|
||||
|
|
@ -247,9 +246,9 @@ def run(self, opts):
|
|||
try:
|
||||
path = path.format(ext)
|
||||
src = os.path.join(ext_dir, os.path.basename(path))
|
||||
subprocess.check_call(['ssh', '-S', control_path, host, 'chmod', '+w', f'"{path}"'])
|
||||
subprocess.check_call(['ssh', '-S', control_path, host, 'chmod', '+w', '"{}"'.format(path)])
|
||||
with open(src, 'rb') as f:
|
||||
p = subprocess.Popen(['ssh', '-S', control_path, host, f'cat - > "{path}"'], stdin=subprocess.PIPE)
|
||||
p = subprocess.Popen(['ssh', '-S', control_path, host, 'cat - > "{}"'.format(path)], stdin=subprocess.PIPE)
|
||||
p.communicate(f.read())
|
||||
if p.wait() != 0:
|
||||
raise SystemExit(1)
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# vim:fileencoding=utf-8
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:fdm=marker:ai
|
||||
# License: GPLv3 Copyright: 2019, Kovid Goyal <kovid at kovidgoyal.net>
|
||||
from __future__ import absolute_import, division, print_function, unicode_literals
|
||||
|
||||
import glob
|
||||
import os
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:fdm=marker:ai
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
# License: GPLv3 Copyright: 2013, Kovid Goyal <kovid at kovidgoyal.net>
|
||||
from __future__ import absolute_import, division, print_function, unicode_literals
|
||||
|
||||
|
||||
# Imports {{{
|
||||
|
|
@ -126,7 +128,7 @@ def parse_index(raw=None): # {{{
|
|||
|
||||
thread_id = url_to_plugin_id(url, deprecated)
|
||||
if thread_id in seen:
|
||||
raise ValueError(f'thread_id for {seen[thread_id]} and {name} is the same: {thread_id}')
|
||||
raise ValueError('thread_id for %s and %s is the same: %s' % (seen[thread_id], name, thread_id))
|
||||
seen[thread_id] = name
|
||||
entry = IndexEntry(name, url, donate, history, uninstall, deprecated, thread_id)
|
||||
yield entry
|
||||
|
|
@ -145,7 +147,7 @@ def load_plugins_index():
|
|||
try:
|
||||
with open(PLUGINS, 'rb') as f:
|
||||
raw = f.read()
|
||||
except OSError as err:
|
||||
except IOError as err:
|
||||
if err.errno == errno.ENOENT:
|
||||
return {}
|
||||
raise
|
||||
|
|
@ -171,13 +173,13 @@ def convert_node(fields, x, names={}, import_data=None):
|
|||
return dict(zip(keys, values))
|
||||
elif name == 'Call':
|
||||
if len(x.args) != 1 and len(x.keywords) != 0:
|
||||
raise TypeError(f'Unsupported function call for fields: {fields}')
|
||||
raise TypeError('Unsupported function call for fields: %s' % (fields,))
|
||||
return tuple(map(conv, x.args))[0]
|
||||
elif name == 'Name':
|
||||
if x.id not in names:
|
||||
if import_data is not None and x.id in import_data[0]:
|
||||
return get_import_data(x.id, import_data[0][x.id], *import_data[1:])
|
||||
raise ValueError(f'Could not find name {x.id} for fields: {fields}')
|
||||
raise ValueError('Could not find name %s for fields: %s' % (x.id, fields))
|
||||
return names[x.id]
|
||||
elif name == 'BinOp':
|
||||
if x.right.__class__.__name__ == 'Str':
|
||||
|
|
@ -186,7 +188,7 @@ def convert_node(fields, x, names={}, import_data=None):
|
|||
return x.right.value
|
||||
elif name == 'Attribute':
|
||||
return conv(getattr(conv(x.value), x.attr))
|
||||
raise TypeError(f'Unknown datatype {x} for fields: {fields}')
|
||||
raise TypeError('Unknown datatype %s for fields: %s' % (x, fields))
|
||||
|
||||
|
||||
Alias = namedtuple('Alias', 'name asname')
|
||||
|
|
@ -219,7 +221,7 @@ def get_import_data(name, mod, zf, names):
|
|||
return convert_node({x}, node.value)
|
||||
if is_module_import:
|
||||
return module
|
||||
raise ValueError(f'Failed to find name: {name!r} in module: {mod!r}')
|
||||
raise ValueError('Failed to find name: %r in module: %r' % (name, mod))
|
||||
else:
|
||||
raise ValueError('Failed to find module: %r' % mod)
|
||||
|
||||
|
|
@ -455,7 +457,7 @@ def fetch_plugins(old_index):
|
|||
|
||||
|
||||
def plugin_to_index(plugin, count):
|
||||
title = '<h3><img src="plugin-icon.png"><a href={} title="Plugin forum thread">{}</a></h3>'.format( # noqa
|
||||
title = '<h3><img src="plugin-icon.png"><a href=%s title="Plugin forum thread">%s</a></h3>' % ( # noqa
|
||||
quoteattr(plugin['thread_url']), escape(plugin['name']))
|
||||
released = datetime(*tuple(map(int, re.split(r'\D', plugin['last_modified'])))[:6]).strftime('%e %b, %Y').lstrip()
|
||||
details = [
|
||||
|
|
@ -476,12 +478,12 @@ def plugin_to_index(plugin, count):
|
|||
block.append('<li>%s</li>' % li)
|
||||
block = '<ul>%s</ul>' % ('\n'.join(block))
|
||||
downloads = ('\xa0<span class="download-count">[%d total downloads]</span>' % count) if count else ''
|
||||
zipfile = '<div class="end"><a href={} title="Download plugin" download={}>Download plugin \u2193</a>{}</div>'.format(
|
||||
zipfile = '<div class="end"><a href=%s title="Download plugin" download=%s>Download plugin \u2193</a>%s</div>' % (
|
||||
quoteattr(plugin['file']), quoteattr(plugin['name'] + '.zip'), downloads)
|
||||
desc = plugin['description'] or ''
|
||||
if desc:
|
||||
desc = '<p>%s</p>' % desc
|
||||
return f'{title}\n{desc}\n{block}\n{zipfile}\n\n'
|
||||
return '%s\n%s\n%s\n%s\n\n' % (title, desc, block, zipfile)
|
||||
|
||||
|
||||
def create_index(index, raw_stats):
|
||||
|
|
@ -524,14 +526,14 @@ def create_index(index, raw_stats):
|
|||
try:
|
||||
with open('index.html', 'rb') as f:
|
||||
oraw = f.read()
|
||||
except OSError:
|
||||
except EnvironmentError:
|
||||
oraw = None
|
||||
if raw != oraw:
|
||||
atomic_write(raw, 'index.html')
|
||||
|
||||
def plugin_stats(x):
|
||||
name, count = x
|
||||
return f'<tr><td>{escape(name)}</td><td>{count}</td></tr>\n'
|
||||
return '<tr><td>%s</td><td>%s</td></tr>\n' % (escape(name), count)
|
||||
|
||||
pstats = list(map(plugin_stats, sorted(stats.items(), reverse=True, key=lambda x:x[1])))
|
||||
stats = '''\
|
||||
|
|
@ -558,7 +560,7 @@ def plugin_stats(x):
|
|||
try:
|
||||
with open('stats.html', 'rb') as f:
|
||||
oraw = f.read()
|
||||
except OSError:
|
||||
except EnvironmentError:
|
||||
oraw = None
|
||||
if raw != oraw:
|
||||
atomic_write(raw, 'stats.html')
|
||||
|
|
@ -572,7 +574,7 @@ def singleinstance():
|
|||
s = _singleinstance = socket.socket(socket.AF_UNIX)
|
||||
try:
|
||||
s.bind(b'\0calibre-plugins-mirror-singleinstance')
|
||||
except OSError as err:
|
||||
except socket.error as err:
|
||||
if getattr(err, 'errno', None) == errno.EADDRINUSE:
|
||||
return False
|
||||
raise
|
||||
|
|
@ -588,7 +590,7 @@ def update_stats():
|
|||
try:
|
||||
with open('stats.json', 'rb') as f:
|
||||
stats = json.load(f)
|
||||
except OSError as err:
|
||||
except EnvironmentError as err:
|
||||
if err.errno != errno.ENOENT:
|
||||
raise
|
||||
if os.geteuid() != 0:
|
||||
|
|
@ -686,7 +688,7 @@ def test_parse(): # {{{
|
|||
new_entries = tuple(parse_index(raw))
|
||||
for i, entry in enumerate(old_entries):
|
||||
if entry != new_entries[i]:
|
||||
print(f'The new entry: {new_entries[i]} != {entry}')
|
||||
print('The new entry: %s != %s' % (new_entries[i], entry))
|
||||
raise SystemExit(1)
|
||||
pool = ThreadPool(processes=20)
|
||||
urls = [e.url for e in new_entries]
|
||||
|
|
@ -703,7 +705,7 @@ def test_parse(): # {{{
|
|||
break
|
||||
new_url, aname = parse_plugin_zip_url(raw)
|
||||
if new_url != full_url:
|
||||
print(f'new url ({aname}): {new_url} != {full_url} for plugin at: {url}')
|
||||
print('new url (%s): %s != %s for plugin at: %s' % (aname, new_url, full_url, url))
|
||||
raise SystemExit(1)
|
||||
|
||||
# }}}
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
|
|
@ -224,7 +225,7 @@ def replace_with_symlinks(self, lang_dir):
|
|||
orig = self.j(self.d(base), r)
|
||||
try:
|
||||
sz = os.stat(orig).st_size
|
||||
except OSError:
|
||||
except EnvironmentError:
|
||||
continue
|
||||
if sz == os.stat(f).st_size and filecmp._do_cmp(f, orig):
|
||||
os.remove(f)
|
||||
|
|
@ -259,7 +260,7 @@ def build_man_pages(self, dest, compress=False):
|
|||
os.environ['ALL_USER_MANUAL_LANGUAGES'] = ' '.join(languages)
|
||||
try:
|
||||
os.makedirs(dest)
|
||||
except OSError:
|
||||
except EnvironmentError:
|
||||
pass
|
||||
jobs = []
|
||||
for l in languages:
|
||||
|
|
@ -267,7 +268,7 @@ def build_man_pages(self, dest, compress=False):
|
|||
[sys.executable, self.j(base, 'build.py'), '--man-pages', l, dest],
|
||||
'\n\n**************** Building translations for: %s' % l)
|
||||
)
|
||||
self.info(f'\tCreating man pages in {dest} for {len(jobs)} languages...')
|
||||
self.info('\tCreating man pages in {} for {} languages...'.format(dest, len(jobs)))
|
||||
subprocess.check_call(jobs[0].cmd)
|
||||
if not parallel_build(jobs[1:], self.info, verbose=False):
|
||||
raise SystemExit(1)
|
||||
|
|
@ -306,4 +307,4 @@ def run(self, opts):
|
|||
subprocess.check_call(
|
||||
'git tag -s v{0} -m "version-{0}"'.format(__version__).split()
|
||||
)
|
||||
subprocess.check_call(f'git push origin v{__version__}'.split())
|
||||
subprocess.check_call('git push origin v{0}'.format(__version__).split())
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=UTF-8:ts=4:sw=4:sta:et:sts=4:ai
|
||||
|
||||
|
||||
__license__ = 'GPL v3'
|
||||
|
|
@ -15,13 +16,17 @@
|
|||
|
||||
def get_opts_from_parser(parser):
|
||||
def do_opt(opt):
|
||||
yield from opt._long_opts
|
||||
yield from opt._short_opts
|
||||
for x in opt._long_opts:
|
||||
yield x
|
||||
for x in opt._short_opts:
|
||||
yield x
|
||||
for o in parser.option_list:
|
||||
yield from do_opt(o)
|
||||
for x in do_opt(o):
|
||||
yield x
|
||||
for g in parser.option_groups:
|
||||
for o in g.option_list:
|
||||
yield from do_opt(o)
|
||||
for x in do_opt(o):
|
||||
yield x
|
||||
|
||||
|
||||
class Kakasi(Command): # {{{
|
||||
|
|
@ -140,7 +145,7 @@ def run(self, opts):
|
|||
try:
|
||||
with open(self.CA_PATH, 'rb') as f:
|
||||
raw = f.read()
|
||||
except OSError as err:
|
||||
except EnvironmentError as err:
|
||||
if err.errno != errno.ENOENT:
|
||||
raise
|
||||
raw = b''
|
||||
|
|
@ -293,7 +298,7 @@ def run(self, opts):
|
|||
except Exception:
|
||||
continue
|
||||
src = src.replace('def ' + func.__name__, 'def replace')
|
||||
imports = [f'from {x.__module__} import {x.__name__}' for x in func.imports]
|
||||
imports = ['from %s import %s' % (x.__module__, x.__name__) for x in func.imports]
|
||||
if imports:
|
||||
src = '\n'.join(imports) + '\n\n' + src
|
||||
function_dict[func.name] = src
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
# License: GPL v3 Copyright: 2019, Eli Schwartz <eschwartz@archlinux.org>
|
||||
|
||||
import os
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
# vim:fileencoding=utf-8
|
||||
# License: GPLv3 Copyright: 2016, Kovid Goyal <kovid at kovidgoyal.net>
|
||||
|
||||
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue