Commit graph

1688 commits

Author SHA1 Message Date
Kovid Goyal
3bece11b09
py3: Misc fixes
Fixes #953 (py3: misc fixes trying to start calibre.gui_launch)
2019-04-02 09:31:37 +05:30
Kovid Goyal
ccb0391844
Change the source for recent user agent data
The old source no longer works
2019-04-01 18:08:29 +05:30
Kovid Goyal
b78e24417a
py3: dont use func_name 2019-04-01 17:01:28 +05:30
Kovid Goyal
67814d2186
py3: misc fixes to get bootstrapping (almost) working 2019-03-31 19:58:00 +05:30
Kovid Goyal
4a3c9ca32f
py3: Fix checking of iso639 translations 2019-03-31 18:38:32 +05:30
Kovid Goyal
e73dbb0f99
py3: Port calls to json.dump() 2019-03-31 18:30:59 +05:30
Kovid Goyal
8b280c03c3
py3: Port RTF metadata module 2019-03-31 13:45:46 +05:30
Kovid Goyal
c33768b35e
Move the python imports test out of the build test module
The build test is designed to run with frozen calibre, where the
source files may not be available. So run it only as part of
the setup test suite.
2019-03-31 09:31:38 +05:30
Kovid Goyal
9b0ed4204d
... 2019-03-26 08:24:54 +05:30
Kovid Goyal
f68c1f2672
DRYer 2019-03-26 08:24:12 +05:30
Kovid Goyal
f66d6f9fcf
Ensure init symbol is exported on py3 2019-03-26 08:14:46 +05:30
Kovid Goyal
6d3fcc853e
Cant use polyglot in plugins_mirror 2019-03-25 17:54:42 +05:30
Kovid Goyal
5918acabad
Misc fixes for the last py3 porting merge 2019-03-25 16:03:25 +05:30
Kovid Goyal
0fcad5f21f
Fix various uses of iteritems missed by dict_fixes 2019-03-25 15:29:08 +05:30
Kovid Goyal
33cea777ac
Merge branch 'py3' of https://github.com/eli-schwartz/calibre 2019-03-25 13:32:21 +05:30
Kovid Goyal
de9d97d688
Port use of BS for comments_html() 2019-03-25 09:52:45 +05:30
Eli Schwartz
4f545af415
autopep8 2019-03-24 01:18:27 -04:00
Eli Schwartz
a623717d96
python3: apply various fixes from python-modernize "dict_six"
Imports were then fixed and consolidated to derive from the internal
polyglot.builtins
2019-03-24 01:18:27 -04:00
Eli Schwartz
13d236f556
python3: make recent_uas build 2019-03-24 00:54:58 -04:00
Eli Schwartz
09e97e5af1
setup: don't report full pathname when creating msgpack'ed resources
The full pathname in the log info was introduced as a side effect of
moving off of pickle.
2019-03-24 00:23:15 -04:00
Kovid Goyal
47ed2df137
Various fixes for the latest py3 commit 2019-03-21 18:33:19 +05:30
Eli Schwartz
d66fdc2df7
autopep8 2019-03-21 01:27:21 -04:00
Eli Schwartz
45f68f552b
python3: make coffee build
add polyglot.socketserver wrapper and fix BaseHTTPServer.HTTPServer
2019-03-21 01:27:21 -04:00
Eli Schwartz
c940d9941a
python3: make kakasi build
Since these are all being opened as non-binary, it makes no sense to
then immediately decode them.
2019-03-21 00:23:43 -04:00
Eli Schwartz
8aceae428e
py3: make setup.py gui work
Apparently compileUi explicitly opens the file/buffer as unicode, not
bytes, and then writes a unicode header string (containing the source
file). So the previous attempt to use BytesIO was wrong in this case.

Also images.qrc is being constructed completely in the right here and
now from a bunch of unicode strings, so just use exactly that.
2019-03-20 23:56:30 -04:00
Kovid Goyal
ae735b2ea3
De-vendor html2text 2019-03-20 14:42:46 +05:30
Kovid Goyal
4c9fb6becd
... 2019-03-15 15:52:12 +05:30
Kovid Goyal
d1e30dfcac
Get rid of some xrange 2019-03-13 15:34:10 +05:30
Kovid Goyal
2d21a8efa2
Some more fixes for the unicode type
Now replaced in all dynamically loaded code. Recipes/metadata
sources/etc. In the case of recipes, since they get compiled by calibre
we simply make the unicode/unichr names available, no need for any
changes to the actual recipes themselves.
2019-03-13 12:04:47 +05:30
Eli Schwartz
cbc42bec23
python3: add unicode/unichr wrappers to polyglot 2019-03-13 06:41:38 +05:30
Kovid Goyal
addf9af51a
Fix generation of trnaslation template for website 2019-03-07 10:22:39 +05:30
Kovid Goyal
9695a6e1cc
Merge branch 'py3' of https://github.com/eli-schwartz/calibre 2019-02-26 08:39:42 +05:30
Eli Schwartz
3258c464e7 Port headless plugin to build on python2/python3
The only thing needed is to port the buildsystem to not pass strings
starting with a "b" as build flags.
2019-02-25 22:04:26 -05:00
Kovid Goyal
7d00c4b0f4
macOS: Respect the system setting for text insertion cursor blink time
This value can be changed with the following commands:

defaults write -g NSTextInsertionPointBlinkPeriodOff -float 500
defaults write -g NSTextInsertionPointBlinkPeriodOn -float 500
2019-02-25 09:16:06 +05:30
Kovid Goyal
e0d5a021dd
Fix #1812891 [calibre-3.38.1 tries to download mathjax during build](https://bugs.launchpad.net/calibre/+bug/1812891) again 2019-02-01 13:38:48 +05:30
Kovid Goyal
a64579338c
Dont download MathJax if it is already present
Fixes #1812891 [calibre-3.38.1 tries to download mathjax  during build](https://bugs.launchpad.net/calibre/+bug/1812891)
2019-01-23 08:03:00 +05:30
Eli Schwartz
18ad088bda build: preserve symlinks when installing
Without this, python's default behavior was to dereference the mathjax
symlinks and install the file contents instead.
2019-01-20 16:53:53 -05:00
Eli Schwartz
82617d3a74 build: add option to symlink to system mathjax installation 2019-01-18 08:40:44 -05:00
Eli Schwartz
6adedd8603 setup/mathjax: fallback to using packed versions of resources
Some mathjax distributions come with the unpacked versions removed as
per the upstream instructions for optimizing an installation. Only use
the unpacked versions if they exist; otherwise fallback on the packed
versions.

Also filter the .woff files to make sure they are actually .woff files,
since they may be files like fonts.dir, fonts.scale, .uuid (created by
xorg-mkfontdir, xorg-mkfontscale, fontconfig).
2019-01-18 08:40:44 -05:00
Kovid Goyal
2ae2d14d62
Rapydscript must be built after mathjax 2019-01-17 09:36:16 +05:30
Kovid Goyal
d08e0a33e1
Implement clean for mathjax builder 2019-01-04 22:15:26 +05:30
Kovid Goyal
98a2f53d90
Remove mathjax from bootstrap since it is now under resources 2019-01-04 21:59:01 +05:30
Kovid Goyal
c184fe37c7
Remove unused code 2019-01-04 21:54:57 +05:30
Kovid Goyal
f78a7dad58
Do not use a compressed archive for mathjax
Improves performance for local clients such as PDF output and the
viewer. Since we have now removed the old unbundled mathjax, the file
count in the resources directory does not go up too much.
2019-01-04 21:53:06 +05:30
Kovid Goyal
e47a2d888b
Update the SWIG bindings for chmlib
Also get rid of the xhm_extra module since
we dont care about searching CHM files and get_lcid can be
implemented in pure python.
2018-12-27 14:44:43 +05:30
Kovid Goyal
bf60631ce0
Avoid the need to use a patched version of MathJax
See https://github.com/mathjax/MathJax/pull/1453
2018-12-17 15:38:14 +05:30
Kovid Goyal
f6ef5f3cf7
Fix crash when compressing mathjax on CI servers 2018-12-13 18:48:23 +05:30
Kovid Goyal
dca2d1a51c
String changes 2018-12-10 08:36:28 +05:30
Kovid Goyal
8018891656
... 2018-10-19 10:30:54 +05:30
Kovid Goyal
17eff935eb
... 2018-10-19 10:16:01 +05:30
Kovid Goyal
79d7658ea3
Check fosshub queue status since the API has now been fixed 2018-10-03 20:30:05 +05:30
Kovid Goyal
e4795fcf88
oops 2018-10-03 08:39:47 +05:30
Kovid Goyal
826119e797
Use new fosshub upload API 2018-10-03 08:39:11 +05:30
Flaviu Tamas
8b9e401f32 Use builtin zlib in py3 2018-10-02 17:01:01 -04:00
Kovid Goyal
01a9fb6687
Fix #1794915 [setup.py build does not work due to UnicodeEncodeError](https://bugs.launchpad.net/calibre/+bug/1794915) 2018-09-28 09:47:35 +05:30
Kovid Goyal
4d4ffcff7c
Revert "Remove use of prints in the setup package since nowadays most systems are UTF-8 based anyway"
This reverts commit f4b3585549.
2018-09-28 09:44:41 +05:30
Flaviu Tamas
1270f32157 Only build monotonic on py2 2018-09-14 13:49:33 -04:00
Flaviu Tamas
46b521dfbf Build hunspell in py3
The verbose old-style intialization is used because GCC 4.8 & MSVC don't
support the nicer `.foo = bar` style of initialization.
2018-09-12 22:41:57 -04:00
Kovid Goyal
be00d40d02
plugins_mirror cannot depend on polyglot 2018-09-10 21:04:25 +05:30
Kovid Goyal
3c10a86c65
setup.py build now runs under python3 2018-09-10 21:02:10 +05:30
Kovid Goyal
e125996ee6
Remove basestring from the setup package 2018-09-10 20:58:01 +05:30
Kovid Goyal
f39f3951f4
dict fixes in the setup package 2018-09-10 20:54:35 +05:30
Kovid Goyal
5bdb5b7c6c
Import fixes for the setup package 2018-09-10 20:44:16 +05:30
Kovid Goyal
cc9a11fba0
Port os.getcwdu to python 3 2018-09-10 20:20:26 +05:30
Kovid Goyal
5234e43f0e
Port future_builtins to polyglot 2018-09-10 20:12:01 +05:30
Kovid Goyal
f4b3585549
Remove use of prints in the setup package since nowadays most systems are UTF-8 based anyway 2018-09-10 19:55:49 +05:30
Kovid Goyal
a9e07efecd
The setup package now imports under python 3 2018-09-10 19:47:53 +05:30
Kovid Goyal
5c6ba41cca
oops 2018-09-10 19:42:20 +05:30
Kovid Goyal
778b3cf74e
Remove unused code 2018-09-10 19:37:42 +05:30
Kovid Goyal
1a928fc497
Remove unused code 2018-09-10 18:41:52 +05:30
Kovid Goyal
b9767b2b92
Move more databases to msgpack instead of pickle 2018-09-10 18:35:19 +05:30
Kovid Goyal
8b3ea8fb83
Replace more use of pickle for databases 2018-09-10 18:26:46 +05:30
Kovid Goyal
d8e24e4a19
Use msgpack instead of pickle for a couple more databases 2018-09-10 17:36:57 +05:30
Kovid Goyal
1d54c2f874
Use msgpack instead of pickle for ISO 639 and 3166 databases 2018-09-10 17:14:03 +05:30
Kovid Goyal
ce377076b9
Some py3 fixes for the setup module 2018-09-10 16:04:47 +05:30
Flaviu Tamas
01fbf09c0b
Undo redundant imports & new-style raises
New-style rasies need to be redone in a py2-friendly way, and the
redudant imports are redudnant and can safely be removed
2018-09-10 12:11:29 +05:30
Flaviu Tamas
655ab21b0b
Fix from __future__ import order 2018-09-10 12:11:29 +05:30
Flaviu Tamas
0889ee85ec
Fix syntax errors
Command used:

futurize --no-diffs -f libfuturize.fixes.fix_print_with_import -f lib2to3.fixes.fix_throw -f lib2to3.fixes.fix_numliterals -f lib2to3.fixes.fix_except -f lib2to3.fixes.fix_exec -f lib2to3.fixes.fix_raise -f lib2to3.fixes.fix_tuple_params -f lib2to3.fixes.fix_ne -j20 -w -n setup recipes src manual setup.py recipes/*.recipe

And manual adjustments of print((...)) -> print(...)
2018-09-10 12:11:28 +05:30
Kovid Goyal
9350caab4e
Fix invocation of setup.py in git hooks 2018-07-27 17:36:25 +05:30
Kovid Goyal
626d5aa1ce
Infrastructure for author name mapping 2018-07-17 15:32:58 +05:30
Joel Goguen
1c25ba0042 Exclude execute from umask
If the umask is set up to exlude execute, the installer will set up
calibre to be unreadable by any user other than the installing user.
When installing as `root`, a starting umask of `0o077` results in the
installer choosing a new umask of `0o033`, which means the calibre
install directory (and child directories) are installed with permissions
`0o744`, so only the installing user can actually use the resulting
calibre install.

To test, start with a system umask of `0o077` and run the installer
script as root:

```
% sudo sh ./setup/linux-installer.sh
Using python executable: /usr/bin/python3
WARNING: Your current umask disallows reading of files by some users, this can cause system breakage when running the installer because of bugs in common system utilities.
Should the installer (f)ix the umask, (i)gnore it or (a)bort [f/i/a Default is abort]: f
umask changed to: 022
Installing to /opt/calibre
Downloading tarball signature securely...
Using previously downloaded calibre-3.26.1-x86_64.txz
Extracting files to /opt/calibre ...
Extracting application files...
Creating symlinks...
	Symlinking /opt/calibre/ebook-edit to /usr/bin/ebook-edit
	Symlinking /opt/calibre/calibre-debug to /usr/bin/calibre-debug
	Symlinking /opt/calibre/web2disk to /usr/bin/web2disk
	Symlinking /opt/calibre/calibre-parallel to /usr/bin/calibre-parallel
	Symlinking /opt/calibre/calibredb to /usr/bin/calibredb
	Symlinking /opt/calibre/lrs2lrf to /usr/bin/lrs2lrf
	Symlinking /opt/calibre/calibre to /usr/bin/calibre
	Symlinking /opt/calibre/ebook-convert to /usr/bin/ebook-convert
	Symlinking /opt/calibre/calibre-server to /usr/bin/calibre-server
	Symlinking /opt/calibre/ebook-viewer to /usr/bin/ebook-viewer
	Symlinking /opt/calibre/calibre-smtp to /usr/bin/calibre-smtp
	Symlinking /opt/calibre/ebook-meta to /usr/bin/ebook-meta
	Symlinking /opt/calibre/ebook-device to /usr/bin/ebook-device
	Symlinking /opt/calibre/fetch-ebook-metadata to /usr/bin/fetch-ebook-metadata
	Symlinking /opt/calibre/ebook-polish to /usr/bin/ebook-polish
	Symlinking /opt/calibre/lrfviewer to /usr/bin/lrfviewer
	Symlinking /opt/calibre/calibre-customize to /usr/bin/calibre-customize
	Symlinking /opt/calibre/lrf2lrs to /usr/bin/lrf2lrs
	Symlinking /opt/calibre/markdown-calibre to /usr/bin/markdown-calibre
Setting up command-line completion...
Installing zsh completion to: /usr/share/zsh/site-functions/_calibre
Installing bash completion to: /usr/share/bash-completion/completions/calibre
Setting up desktop integration...
Creating un-installer: /usr/bin/calibre-uninstall
Run "calibre" to start calibre
```

And the resulting directory:
```
% ls -ld /opt/calibre
drwxr-xr-x. 5 root root 4.0K Jun 22 10:53 /opt/calibre/
```

Also verify the Python version, starting with no `/opt/calibre` and a system umask of `0o077`:
```
% sudo python3 ./setup/linux-installer.py
WARNING: Your current umask disallows reading of files by some users, this can cause system breakage when running the installer because of bugs in common system utilities.
Should the installer (f)ix the umask, (i)gnore it or (a)bort [f/i/a Default is abort]: f
umask changed to: 022
Installing to /opt/calibre
Downloading tarball signature securely...
Using previously downloaded calibre-3.26.1-x86_64.txz
Extracting files to /opt/calibre ...
Extracting application files...
Creating symlinks...
	Symlinking /opt/calibre/calibre-customize to /usr/bin/calibre-customize
	Symlinking /opt/calibre/calibre-smtp to /usr/bin/calibre-smtp
	Symlinking /opt/calibre/calibredb to /usr/bin/calibredb
	Symlinking /opt/calibre/calibre to /usr/bin/calibre
	Symlinking /opt/calibre/ebook-polish to /usr/bin/ebook-polish
	Symlinking /opt/calibre/ebook-meta to /usr/bin/ebook-meta
	Symlinking /opt/calibre/calibre-server to /usr/bin/calibre-server
	Symlinking /opt/calibre/markdown-calibre to /usr/bin/markdown-calibre
	Symlinking /opt/calibre/fetch-ebook-metadata to /usr/bin/fetch-ebook-metadata
	Symlinking /opt/calibre/lrf2lrs to /usr/bin/lrf2lrs
	Symlinking /opt/calibre/calibre-parallel to /usr/bin/calibre-parallel
	Symlinking /opt/calibre/ebook-convert to /usr/bin/ebook-convert
	Symlinking /opt/calibre/ebook-viewer to /usr/bin/ebook-viewer
	Symlinking /opt/calibre/web2disk to /usr/bin/web2disk
	Symlinking /opt/calibre/calibre-debug to /usr/bin/calibre-debug
	Symlinking /opt/calibre/ebook-device to /usr/bin/ebook-device
	Symlinking /opt/calibre/lrfviewer to /usr/bin/lrfviewer
	Symlinking /opt/calibre/ebook-edit to /usr/bin/ebook-edit
	Symlinking /opt/calibre/lrs2lrf to /usr/bin/lrs2lrf
Setting up command-line completion...
Installing zsh completion to: /usr/share/zsh/site-functions/_calibre
Installing bash completion to: /usr/share/bash-completion/completions/calibre
Setting up desktop integration...
Creating un-installer: /usr/bin/calibre-uninstall
Run "calibre" to start calibre

% ls -ld /opt/calibre
drwxr-xr-x. 5 root root 4.0K Jun 22 10:55 /opt/calibre/
```

After each, verify calibre starts as a non-root user.
2018-06-22 10:49:26 -07:00
Kovid Goyal
4df998fdd2
Ignore 404s on private bugs 2018-06-18 13:24:33 +05:30
Kovid Goyal
e8fdd7a83c
Migrate git hooks to py3 2018-06-12 18:01:55 +05:30
Kovid Goyal
cd7e8891bd
Fix python invocation in git hooks 2018-06-10 07:33:54 +05:30
Kovid Goyal
73029303d6
Dont use a set literal in the installer since there are apparently people still running linux installations with python 2.6 2018-06-09 07:05:03 +05:30
Kovid Goyal
499578beaa
Try to get unicode_names compiling on ancient compilers 2018-05-01 11:31:21 +05:30
Kovid Goyal
eaffdbb884
... 2018-05-01 11:18:40 +05:30
Kovid Goyal
5d95d13935
Edit Book: Fix names for some control characters not being displayed in the status bar
Uses the unicode names database I created for kitty. Much more
comprehensive than the one in ICU.
2018-05-01 10:23:50 +05:30
Kovid Goyal
f04b5eea20
Remove use of command -v from linux-installer.sh 2018-04-30 08:29:59 +05:30
Kovid Goyal
7a150c065e
Add a shebang for linux-installer.sh 2018-04-29 15:11:46 +05:30
Kovid Goyal
54c81eec15
Simplify passing arguments to linux-installer.py 2018-04-29 14:57:59 +05:30
Kovid Goyal
db936049cd
Add a shell script wrapper for the linux installer
Needed because Ubuntu 18.04 has decided, in its infinite wisdom to
have no default python executable, which makes it impossible to run
polyglot python scripts.
2018-04-28 10:05:18 +05:30
Kovid Goyal
9679a1c4f3
pep8 2018-04-28 08:52:46 +05:30
Kovid Goyal
2373a38005
Linux installer: Fix umask question not working with the recommended install command because stdin is a pipe 2018-04-14 15:44:02 +05:30
Kovid Goyal
baee2d2546
Content server: When updating interface data do not transmit translations if they have not been changed. Saves ~30-60KB bandwidth when using non-English interface language. 2018-03-24 10:50:24 +05:30
Kovid Goyal
73cd57bebb
DRYer 2018-03-10 09:07:38 +05:30
Kovid Goyal
53e5d1a748
Fix #1754698 [SIP_DIR no longer works to specify pyqt_sip_dir](https://bugs.launchpad.net/calibre/+bug/1754698) 2018-03-10 09:04:54 +05:30
Kovid Goyal
2beeb537b4
Ensure no spaces around fosshub API key 2018-03-09 16:41:00 +05:30
Kovid Goyal
5697f8f282
More robust timeout retry handling 2018-02-02 15:43:55 +05:30
Kovid Goyal
3dfd846b7b
Retry after sleeping on URL fetch timeout in plugins mirror script 2018-02-02 13:14:04 +05:30