Commits · MAsad789565/web-search-api

Update documents/query_results_extractor.py

8aa7160
verified

MAsad789565 commited on Mar 27

Update apis/search_api.py

cde1a76
verified

MAsad789565 commited on Mar 26

Update apis/search_api.py

bb23eff
verified

MAsad789565 commited on Mar 26

Update networks/google_searcher.py

e1afdc5
verified

MAsad789565 commited on Mar 26

Update networks/google_searcher.py

caebafc
verified

MAsad789565 commited on Mar 26

Update networks/filepath_converter.py

e35d7eb
verified

MAsad789565 commited on Mar 26

Update networks/webpage_fetcher.py

38e2473
verified

MAsad789565 commited on Mar 26

Update documents/query_results_extractor.py

e789120
verified

MAsad789565 commited on Mar 26

Update documents/query_results_extractor.py

fa9ef68
verified

MAsad789565 commited on Mar 26

:zap: [Enhance] Add ignore classes for wikipedia.org

dd996c1

Hansimov commited on Jan 11

:zap: [Enhance] Loop multiple conditions for extracting abstract

a315628

Hansimov commited on Jan 11

:boom: [Fix] PermissionError: [Errno 13] Permission denied: /app/files

ea49401

Hansimov commited on Jan 11

:gem: [Feature] Sync to Hugginface space, and run in docker

d0a4f07

Hansimov commited on Jan 11

:boom: [Fix] SearchAPIApp: incorrect order of extracted contents to urls

eb0ce75

Hansimov commited on Jan 11

:pencil: [Config] Generate requirements.txt

d3368e0

Hansimov commited on Jan 11

:recycle: [Refactor] Replace output_path with html_path to avoid confuse

8c0b736

Hansimov commited on Jan 11

:gem: [Feature] SearchAPIApp: Concurrent fetch urls and extract contents

f234ce3

Hansimov commited on Jan 11

:boom: [Fix] WebpageFetcher: raise timeout when request.get hangs

bce51d4

Hansimov commited on Jan 11

:boom: [Fix] WebpageContentExtractor: UnicodeDecodeError

cff1afc

Hansimov commited on Jan 11

:recycle: [Refactor] QueryResultsExtractor: prettify logging

0acc824

Hansimov commited on Jan 11

:zap: [Enhance] BatchWebpageFetcher: return url_and_output_path_list

4591d96

Hansimov commited on Jan 11

:gem: [Feature] New BatchWebpageContentExtractor: Extract webpage content from multiple html_paths concurrently

1db460d

Hansimov commited on Jan 11

:gem: [Feature] New BatchWebpageFetcher: Fetch multiple urls concurrently

e92817a

Hansimov commited on Jan 11

:boom: [Fix] Duplicated query_results in response JSON when passing multiple queries

876e441

Hansimov commited on Jan 11

:zap: [Enhance] WebpageContentExtractor: Escape dash, and ignore

c7c538d

Hansimov commited on Jan 10

:zap: [Enhance] ignore classes pattern, especially for 163.com

3dda344

Hansimov commited on Jan 10

:zap: [Enhance] Rename HTMLFetcher to WebpageFetcher, and add output_parent param

62ee9e4

Hansimov commited on Jan 10

:zap: [Enhance] SearchAPIApp: overwrite param for query and webpage HTML

9fb4731

Hansimov commited on Jan 10

:recycle: [Refactor] WebpageContentExtractor: Separate html and markdown processing

a636bcb

Hansimov commited on Jan 10

:recycle: [Refactor] Move hardcoded consts to network_configs

af2c647

Hansimov commited on Jan 10

:zap: [Enhance] HTMLFetcher and GoogleSearcher: support cache with overwrite, and ignore host

cf4c3f8

Hansimov commited on Jan 10

:gem: [Feature] SearchAPIApp: New extract_content param

4d3e890

Hansimov commited on Jan 10

:gem: [Feature] New WebpageContentExtractor: extract webpage content as clean markdown

e773696

Hansimov commited on Jan 10

:recycle: [Refactor] HTMLFetcher: replace save_path with output_path

7d44e75

Hansimov commited on Jan 10

:gem: [Feature] Enable SearchAPIApp: /queries_to_search_results

138c09e

Hansimov commited on Jan 10

:zap: [Enhance] GoogleSearcher: Add params of result_sum and safe

8bf48d8

Hansimov commited on Jan 10

:recycle: [Refactor] Rename SearchResultsExtractor to QueryResultsExtractor, and store results

0f6452f

Hansimov commited on Jan 10

:zap: [Enhance] FilepathConverter: New parent param when init

f9c42cf

Hansimov commited on Jan 10

:gem: [Feature] New HTMLFetcher: download url to local html file

b259fec

Hansimov commited on Jan 7

:gem: [Feature] New FilepathConverter: convert urls and queries to valid file path

64a0dbf

Hansimov commited on Jan 7

:recycle: [Refactor] Move header constructor, and prettier logging

e448a74

Hansimov commited on Jan 6

:gem: [Feature] SearchResultsExtractor: related questions

f150f6b

Hansimov commited on Jan 6

:gem: [Feature] New SearchResultsExtractor: title, site, link, abstract

ef3de03

Hansimov commited on Jan 6

:pencil: [Doc] Readme and git ignore

d6015f4

Hansimov commited on Jan 6

:gem: [Feature] New Enver and Logger

f2ec1a1

Hansimov commited on Jan 6

:gem: [Feature] New GoogleSearcher: Enable google search with query

6cf0820

Hansimov commited on Jan 6

Commit History

Update documents/query_results_extractor.py 8aa7160 verified

Update apis/search_api.py cde1a76 verified

Update apis/search_api.py bb23eff verified

Update networks/google_searcher.py e1afdc5 verified

Update networks/google_searcher.py caebafc verified

Update networks/filepath_converter.py e35d7eb verified

Update networks/webpage_fetcher.py 38e2473 verified

Update documents/query_results_extractor.py e789120 verified

Update documents/query_results_extractor.py fa9ef68 verified

:zap: [Enhance] Add ignore classes for wikipedia.org dd996c1

:zap: [Enhance] Loop multiple conditions for extracting abstract a315628

:boom: [Fix] PermissionError: [Errno 13] Permission denied: /app/files ea49401

:gem: [Feature] Sync to Hugginface space, and run in docker d0a4f07

:boom: [Fix] SearchAPIApp: incorrect order of extracted contents to urls eb0ce75

:pencil: [Config] Generate requirements.txt d3368e0

:recycle: [Refactor] Replace output_path with html_path to avoid confuse 8c0b736

:gem: [Feature] SearchAPIApp: Concurrent fetch urls and extract contents f234ce3

:boom: [Fix] WebpageFetcher: raise timeout when request.get hangs bce51d4

:boom: [Fix] WebpageContentExtractor: UnicodeDecodeError cff1afc

:recycle: [Refactor] QueryResultsExtractor: prettify logging 0acc824

:zap: [Enhance] BatchWebpageFetcher: return url_and_output_path_list 4591d96

:gem: [Feature] New BatchWebpageContentExtractor: Extract webpage content from multiple html_paths concurrently 1db460d

:gem: [Feature] New BatchWebpageFetcher: Fetch multiple urls concurrently e92817a

:boom: [Fix] Duplicated query_results in response JSON when passing multiple queries 876e441

:zap: [Enhance] WebpageContentExtractor: Escape dash, and ignore c7c538d

:zap: [Enhance] ignore classes pattern, especially for 163.com 3dda344

:zap: [Enhance] Rename HTMLFetcher to WebpageFetcher, and add output_parent param 62ee9e4

:zap: [Enhance] SearchAPIApp: overwrite param for query and webpage HTML 9fb4731

:recycle: [Refactor] WebpageContentExtractor: Separate html and markdown processing a636bcb

:recycle: [Refactor] Move hardcoded consts to network_configs af2c647

:zap: [Enhance] HTMLFetcher and GoogleSearcher: support cache with overwrite, and ignore host cf4c3f8

:gem: [Feature] SearchAPIApp: New extract_content param 4d3e890

:gem: [Feature] New WebpageContentExtractor: extract webpage content as clean markdown e773696

:recycle: [Refactor] HTMLFetcher: replace save_path with output_path 7d44e75

:gem: [Feature] Enable SearchAPIApp: /queries_to_search_results 138c09e

:zap: [Enhance] GoogleSearcher: Add params of result_sum and safe 8bf48d8

:recycle: [Refactor] Rename SearchResultsExtractor to QueryResultsExtractor, and store results 0f6452f

:zap: [Enhance] FilepathConverter: New parent param when init f9c42cf

:gem: [Feature] New HTMLFetcher: download url to local html file b259fec

:gem: [Feature] New FilepathConverter: convert urls and queries to valid file path 64a0dbf

:recycle: [Refactor] Move header constructor, and prettier logging e448a74

:gem: [Feature] SearchResultsExtractor: related questions f150f6b

:gem: [Feature] New SearchResultsExtractor: title, site, link, abstract ef3de03

:pencil: [Doc] Readme and git ignore d6015f4

:gem: [Feature] New Enver and Logger f2ec1a1

:gem: [Feature] New GoogleSearcher: Enable google search with query 6cf0820

Update documents/query_results_extractor.py

8aa7160
verified

Update apis/search_api.py

cde1a76
verified

Update apis/search_api.py

bb23eff
verified

Update networks/google_searcher.py

e1afdc5
verified

Update networks/google_searcher.py

caebafc
verified

Update networks/filepath_converter.py

e35d7eb
verified

Update networks/webpage_fetcher.py

38e2473
verified

Update documents/query_results_extractor.py

e789120
verified

Update documents/query_results_extractor.py

fa9ef68
verified

:zap: [Enhance] Add ignore classes for wikipedia.org

dd996c1

:zap: [Enhance] Loop multiple conditions for extracting abstract

a315628

:boom: [Fix] PermissionError: [Errno 13] Permission denied: /app/files

ea49401

:gem: [Feature] Sync to Hugginface space, and run in docker

d0a4f07

:boom: [Fix] SearchAPIApp: incorrect order of extracted contents to urls

eb0ce75

:pencil: [Config] Generate requirements.txt

d3368e0

:recycle: [Refactor] Replace output_path with html_path to avoid confuse

8c0b736

:gem: [Feature] SearchAPIApp: Concurrent fetch urls and extract contents

f234ce3

:boom: [Fix] WebpageFetcher: raise timeout when request.get hangs

bce51d4

:boom: [Fix] WebpageContentExtractor: UnicodeDecodeError

cff1afc

:recycle: [Refactor] QueryResultsExtractor: prettify logging

0acc824

:zap: [Enhance] BatchWebpageFetcher: return url_and_output_path_list

4591d96

:gem: [Feature] New BatchWebpageContentExtractor: Extract webpage content from multiple html_paths concurrently

1db460d

:gem: [Feature] New BatchWebpageFetcher: Fetch multiple urls concurrently

e92817a

:boom: [Fix] Duplicated query_results in response JSON when passing multiple queries

876e441

:zap: [Enhance] WebpageContentExtractor: Escape dash, and ignore

c7c538d

:zap: [Enhance] ignore classes pattern, especially for 163.com

3dda344

:zap: [Enhance] Rename HTMLFetcher to WebpageFetcher, and add output_parent param

62ee9e4

:zap: [Enhance] SearchAPIApp: overwrite param for query and webpage HTML

9fb4731

:recycle: [Refactor] WebpageContentExtractor: Separate html and markdown processing

a636bcb

:recycle: [Refactor] Move hardcoded consts to network_configs

af2c647

:zap: [Enhance] HTMLFetcher and GoogleSearcher: support cache with overwrite, and ignore host

cf4c3f8

:gem: [Feature] SearchAPIApp: New extract_content param

4d3e890

:gem: [Feature] New WebpageContentExtractor: extract webpage content as clean markdown

e773696

:recycle: [Refactor] HTMLFetcher: replace save_path with output_path

7d44e75

:gem: [Feature] Enable SearchAPIApp: /queries_to_search_results

138c09e

:zap: [Enhance] GoogleSearcher: Add params of result_sum and safe

8bf48d8

:recycle: [Refactor] Rename SearchResultsExtractor to QueryResultsExtractor, and store results

0f6452f

:zap: [Enhance] FilepathConverter: New parent param when init

f9c42cf

:gem: [Feature] New HTMLFetcher: download url to local html file

b259fec

:gem: [Feature] New FilepathConverter: convert urls and queries to valid file path

64a0dbf

:recycle: [Refactor] Move header constructor, and prettier logging

e448a74

:gem: [Feature] SearchResultsExtractor: related questions

f150f6b

:gem: [Feature] New SearchResultsExtractor: title, site, link, abstract

ef3de03

:pencil: [Doc] Readme and git ignore

d6015f4

:gem: [Feature] New Enver and Logger

f2ec1a1

:gem: [Feature] New GoogleSearcher: Enable google search with query

6cf0820