Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/data loader #6

Merged
merged 41 commits into from
Sep 11, 2020
Merged

Feature/data loader #6

merged 41 commits into from
Sep 11, 2020

Conversation

inmoonlight
Copy link
Member

@inmoonlight inmoonlight commented Sep 6, 2020

Pull Request

레파지토리에 기여해주셔서 감사드립니다.

해당 PR을 제출하기 전에 아래 사항이 완료되었는지 확인 부탁드립니다:

  • 작성한 코드가 어떤 에러나 경고없이 빌드가 되었나요?
  • 충분한 테스트를 수행하셨나요?

1. 해당 PR은 어떤 내용인가요?

  • minor bug fix
    • tokenizer 쪽에 <eos> <bos> <pad> 를 reserve하지 않았어서 다시 학습하여 올립니다
    • 불필요한 코드는 제거하였습니다
  • main contributions
    • raw text를 읽어서 위에서 학습한 tokenizer 를 통해 encoding, padding
    • 향후 training에 사용할 data loader

2. PR과 관련된 이슈가 있나요?

#1

@inmoonlight inmoonlight force-pushed the feature/data-loader branch 2 times, most recently from f8d63f4 to 609d798 Compare September 8, 2020 15:54
@codecov
Copy link

codecov bot commented Sep 8, 2020

Codecov Report

Merging #6 into master will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##            master        #6    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files            5         5            
  Lines           61       183   +122     
==========================================
+ Hits            61       183   +122     
Flag Coverage Δ
#tests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
tests/__init__.py 100.00% <ø> (ø)
src/dataset.py 100.00% <100.00%> (ø)
src/utils.py 100.00% <100.00%> (ø)
tests/test_dataset.py 100.00% <100.00%> (ø)
tests/test_utils.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9728c46...8bcbf91. Read the comment docs.

src/dataset.py Outdated Show resolved Hide resolved
src/utils.py Outdated Show resolved Hide resolved
tests/test_utils.py Outdated Show resolved Hide resolved
Copy link
Member

@ssaru ssaru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@inmoonlight
코멘트 완료했습니다.
수고하셨습니다. 🙇‍♂️

@ssaru
Copy link
Member

ssaru commented Sep 9, 2020

@aisolab 보섭님도 한번 봐주시지요~

@ssaru ssaru requested a review from seopbo September 9, 2020 14:47
Copy link
Member

@ssaru ssaru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@inmoonlight Approve했습니다.
고생하셨습니다.

src/utils.py Outdated Show resolved Hide resolved
@inmoonlight inmoonlight merged commit e4ed23f into master Sep 11, 2020
@inmoonlight inmoonlight deleted the feature/data-loader branch September 13, 2020 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants