will/github-changelog-generator
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #76 from sneal/issue-71-add-max-issues-arg

Browse Source 
Add --max-issues argument to limit requests
This commit is contained in:
Petr Korolev 2015-03-24 11:26:14 +02:00
commit d6cbfe3c38
3 changed files with 79 additions and 70 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
README.md
View File

@ -96,6 +96,7 @@ Type `github_changelog_generator --help` for detailed usage.
--[no-]compare-link Include compare link between older version and newer version. Default is true --[no-]compare-link Include compare link between older version and newer version. Default is true
--include-labels x,y,z Issues only with that labels will be included to changelog. Default is 'bug,enhancement' --include-labels x,y,z Issues only with that labels will be included to changelog. Default is 'bug,enhancement'
--exclude-labels x,y,z Issues with that labels will be always excluded from changelog. Default is 'duplicate,question,invalid,wontfix' --exclude-labels x,y,z Issues with that labels will be always excluded from changelog. Default is 'duplicate,question,invalid,wontfix'
--max-issues [NUMBER] Max number of issues to fetch from GitHub. Default is unlimited.
--github-site [URL] The Enterprise Github site on which your project is hosted. --github-site [URL] The Enterprise Github site on which your project is hosted.
--github-api [URL] The enterprise endpoint to use for your Github API. --github-api [URL] The enterprise endpoint to use for your Github API.
-v, --version Print version number -v, --version Print version number
@ -186,6 +187,15 @@ I think, that GitHub Releases is more for end-users.
But `CHANGELOG.md` could stay in the repo for developers with detailed list of changes. But `CHANGELOG.md` could stay in the repo for developers with detailed list of changes.
And it's nothing bad to combine GitHub Releases and `CHANGELOG.md` file together in that manner. And it's nothing bad to combine GitHub Releases and `CHANGELOG.md` file together in that manner.
- ***I received a warning: GitHub API rate limit exceed, what does this mean?***
GitHub [limits the number of API requests](https://developer.github.com/v3/#rate-limiting) you can make in an hour. You can make up to 5,000 requests per hour. For unauthenticated requests, the rate limit allows you to make up to 60 requests per hour. Unauthenticated requests are associated with your IP address, and not the user making requests.
If you're seeing this warning:
1. Make sure you're providing an OAuth token so you're not anonymously making requests. This will increase the number of requests from 60 to 5000 per hour.
2. You probably have a large repo with lots of issues/PRs. You can use the `--max-issues NUM` argument to limit the number of issues that are pulled back. For example: `--max-issues 1000`
## Contributing ## Contributing
1. Create an issue to discuss about your idea 1. Create an issue to discuss about your idea

135
lib/github_changelog_generator.rb
View File

@ -15,6 +15,8 @@ module GitHubChangelogGenerator
attr_accessor :options, :all_tags, :github attr_accessor :options, :all_tags, :github
PER_PAGE_NUMBER = 30 PER_PAGE_NUMBER = 30
GH_RATE_LIMIT_EXCEEDED_MSG = 'Warning: GitHub API rate limit exceed (5000 per hour), change log may not ' +
'contain some issues. You can limit the number of issues fetched using the `--max-issues NUM` argument'
def initialize def initialize
@ -27,11 +29,11 @@ module GitHubChangelogGenerator
github_options[:endpoint] = options[:github_endpoint] unless options[:github_endpoint].nil? github_options[:endpoint] = options[:github_endpoint] unless options[:github_endpoint].nil?
github_options[:site] = options[:github_endpoint] unless options[:github_site].nil? github_options[:site] = options[:github_endpoint] unless options[:github_site].nil?
begin begin
@github = Github.new github_options @github = Github.new github_options
rescue rescue
puts "Warning: GitHub API rate limit exceed (5000 per hour), change log may not contain some issues.".yellow puts GH_RATE_LIMIT_EXCEEDED_MSG.yellow
end end
@generator = Generator.new(@options) @generator = Generator.new(@options)
@ -92,11 +94,11 @@ rescue
issue[:actual_date] = issue[:closed_at] issue[:actual_date] = issue[:closed_at]
else else
begin begin
begin begin
commit = @github.git_data.commits.get @options[:user], @options[:project], event[:commit_id] commit = @github.git_data.commits.get @options[:user], @options[:project], event[:commit_id]
rescue rescue
puts "Warning: GitHub API rate limit exceed (5000 per hour), change log may not contain some issues.".yellow puts GH_RATE_LIMIT_EXCEEDED_MSG.yellow
end end
issue[:actual_date] = commit[:author][:date] issue[:actual_date] = commit[:author][:date]
rescue rescue
puts "Warning: can't fetch commit #{event[:commit_id]} probably it referenced from another repo." puts "Warning: can't fetch commit #{event[:commit_id]} probably it referenced from another repo."
@ -119,20 +121,18 @@ rescue
print "Fetching merged dates...\r" print "Fetching merged dates...\r"
end end
pull_requests = [] pull_requests = []
begin begin
response = @github.pull_requests.list @options[:user], @options[:project], :state => 'closed' response = @github.pull_requests.list @options[:user], @options[:project], :state => 'closed'
page_i = 0 page_i = 0
response.each_page do |page| response.each_page do |page|
page_i += PER_PAGE_NUMBER page_i += PER_PAGE_NUMBER
count_pages = response.count_pages count_pages = response.count_pages
print "Fetching merged dates... #{page_i}/#{count_pages * PER_PAGE_NUMBER}\r" print "Fetching merged dates... #{page_i}/#{count_pages * PER_PAGE_NUMBER}\r"
pull_requests.concat(page) pull_requests.concat(page)
end end
rescue rescue
puts "Warning: GitHub API rate limit exceed (5000 per hour), change log may not contain some issues.".yellow puts GH_RATE_LIMIT_EXCEEDED_MSG.yellow
end end
print " \r" print " \r"
@ -290,11 +290,9 @@ rescue
end end
def is_megred(number) def is_megred(number)
begin @github.pull_requests.merged? @options[:user], @options[:project], number
@github.pull_requests.merged? @options[:user], @options[:project], number rescue
rescue puts GH_RATE_LIMIT_EXCEEDED_MSG.yellow
puts "Warning: GitHub API rate limit exceed (5000 per hour), change log may not contain some issues.".yellow
end
end end
def get_all_tags def get_all_tags
@ -306,23 +304,21 @@ rescue
tags = [] tags = []
begin begin
response = @github.repos.tags @options[:user], @options[:project] response = @github.repos.tags @options[:user], @options[:project]
page_i = 0 page_i = 0
count_pages = response.count_pages count_pages = response.count_pages
response.each_page do |page| response.each_page do |page|
page_i += PER_PAGE_NUMBER page_i += PER_PAGE_NUMBER
print "Fetching tags... #{page_i}/#{count_pages * PER_PAGE_NUMBER}\r" print "Fetching tags... #{page_i}/#{count_pages * PER_PAGE_NUMBER}\r"
tags.concat(page) tags.concat(page)
end end
print " \r" print " \r"
if @options[:verbose] if @options[:verbose]
puts "Found #{tags.count} tags" puts "Found #{tags.count} tags"
end end
rescue rescue
puts "Warning: GitHub API rate limit exceed (5000 per hour), change log may not contain some issues.".yellow puts GH_RATE_LIMIT_EXCEEDED_MSG.yellow
end end
tags tags
end end
@ -536,11 +532,11 @@ rescue
return @tag_times_hash[tag_name['name']] return @tag_times_hash[tag_name['name']]
end end
begin begin
github_git_data_commits_get = @github.git_data.commits.get @options[:user], @options[:project], tag_name['commit']['sha'] github_git_data_commits_get = @github.git_data.commits.get @options[:user], @options[:project], tag_name['commit']['sha']
rescue rescue
puts "Warning: GitHub API rate limit exceed (5000 per hour), change log may not contain some issues.".yellow puts GH_RATE_LIMIT_EXCEEDED_MSG.yellow
end end
time_string = github_git_data_commits_get['committer']['date'] time_string = github_git_data_commits_get['committer']['date']
@tag_times_hash[tag_name['name']] = Time.parse(time_string) @tag_times_hash[tag_name['name']] = Time.parse(time_string)
end end
@ -588,20 +584,19 @@ rescue
end end
issues = [] issues = []
begin begin
response = @github.issues.list user: @options[:user], repo: @options[:project], state: 'closed', filter: 'all', labels: nil response = @github.issues.list user: @options[:user], repo: @options[:project], state: 'closed', filter: 'all', labels: nil
page_i = 0 page_i = 0
count_pages = response.count_pages count_pages = response.count_pages
response.each_page do |page| response.each_page do |page|
page_i += PER_PAGE_NUMBER page_i += PER_PAGE_NUMBER
print "Fetching issues... #{page_i}/#{count_pages * PER_PAGE_NUMBER}\r" print "Fetching issues... #{page_i}/#{count_pages * PER_PAGE_NUMBER}\r"
issues.concat(page) issues.concat(page)
end break if @options[:max_issues] && issues.length >= @options[:max_issues]
rescue end
puts "Warning: GitHub API rate limit exceed (5000 per hour), change log may not contain some issues.".yellow rescue
end puts GH_RATE_LIMIT_EXCEEDED_MSG.yellow
end
print " \r" print " \r"
@ -643,11 +638,11 @@ rescue
issues.each_slice(max_thread_number) { |issues_slice| issues.each_slice(max_thread_number) { |issues_slice|
issues_slice.each { |issue| issues_slice.each { |issue|
threads << Thread.new { threads << Thread.new {
begin begin
obj = @github.issues.events.list user: @options[:user], repo: @options[:project], issue_number: issue['number'] obj = @github.issues.events.list user: @options[:user], repo: @options[:project], issue_number: issue['number']
rescue rescue
puts "Warning: GitHub API rate limit exceed (5000 per hour), change log may not contain some issues.".yellow puts GH_RATE_LIMIT_EXCEEDED_MSG.yellow
end end
issue[:events] = obj.body issue[:events] = obj.body
print "Fetching events for issues and PR: #{i+1}/#{@issues.count + @pull_requests.count}\r" print "Fetching events for issues and PR: #{i+1}/#{@issues.count + @pull_requests.count}\r"
i +=1 i +=1

4
lib/github_changelog_generator/parser.rb
View File

@ -24,6 +24,7 @@ module GitHubChangelogGenerator
:enhancement_prefix => '**Implemented enhancements:**', :enhancement_prefix => '**Implemented enhancements:**',
:author => true, :author => true,
:filter_issues_by_milestone => true, :filter_issues_by_milestone => true,
:max_issues => nil,
:compare_link => true, :compare_link => true,
:unreleased => true, :unreleased => true,
:unreleased_label => 'Unreleased', :unreleased_label => 'Unreleased',
@ -83,6 +84,9 @@ module GitHubChangelogGenerator
opts.on('--exclude-labels x,y,z', Array, 'Issues with that labels will be always excluded from changelog. Default is \'duplicate,question,invalid,wontfix\'') do |list| opts.on('--exclude-labels x,y,z', Array, 'Issues with that labels will be always excluded from changelog. Default is \'duplicate,question,invalid,wontfix\'') do |list|
options[:exclude_labels] = list options[:exclude_labels] = list
end end
opts.on('--max-issues [NUMBER]', Integer, 'Max number of issues to fetch from GitHub. Default is unlimited') do |max|
options[:max_issues] = max
end
opts.on('--github-site [URL]', 'The Enterprise Github site on which your project is hosted.') do |last| opts.on('--github-site [URL]', 'The Enterprise Github site on which your project is hosted.') do |last|
options[:github_site] = last options[:github_site] = last
end end