Backup_Repos/FanFicFare
1
0
Fork 0
mirror of https://github.com/JimmXinu/FanFicFare.git synced 2026-05-02 03:48:40 +02:00
Code Issues Projects Releases Wiki Activity

Fix for Get URLs from Page when poor HTML--double soup like base_adapter.

Browse source
This commit is contained in:
Jim Miller 2016-10-31 12:42:41 -05:00
parent 96da2eab89
commit f8132eb14b
1 changed files with 3 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
fanficfare/geturls.py
View file

@ -86,7 +86,9 @@ def get_urls_from_html(data,url=None,configuration=None,normalize=False,restrict
if not configuration:
configuration = Configuration(["test1.com"],"EPUB",lightweight=True)
soup = BeautifulSoup(data,"html5lib")
## soup and re-soup because BS4/html5lib is more forgiving of
## incorrectly nested tags that way.
soup = BeautifulSoup(unicode(BeautifulSoup(data,"html5lib")),"html5lib")
if restrictsearch:
soup = soup.find(*restrictsearch)
#logger.debug("restrict search:%s"%soup)