Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get language from .gitattributes file #14833

Closed
wants to merge 7 commits into from

Conversation

a1012112796
Copy link
Member

@a1012112796 a1012112796 commented Feb 28, 2021

rules:
linguist-language= attribute to an lang
linguist-vendored attribute to vendor or un-vendor path

ref:
https://stackoverflow.com/questions/40659265/using-gitattributes-for-linguist-examples

fixes #14786

rules:
linguist-language=<lang> attribute to an lang
linguist-vendored attribute to vendor or un-vendor path

ref:
https://stackoverflow.com/questions/40659265/using-gitattributes-for-linguist-examples

Signed-off-by: a1012112796 <1012112796@qq.com>
@a1012112796 a1012112796 added the type/enhancement An improvement of existing functionality label Feb 28, 2021
Copy link
Member

@noerw noerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs need to explain this feature & its syntax, similar to https://github.com/github/linguist/blob/master/docs/overrides.md

fixes #14786

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Feb 28, 2021
@zeripath
Copy link
Contributor

Why aren't you using check-attr and already written function that uses it?

@zeripath
Copy link
Contributor

NAME
       git-check-attr - Display gitattributes information

SYNOPSIS
       git check-attr [-a | --all | <attr>...] [--] <pathname>...
       git check-attr --stdin [-z] [-a | --all | <attr>...]

DESCRIPTION
       For every pathname, this command will list if each attribute is unspecified, set, or unset as a
       gitattribute on that pathname.
// CheckAttributeOpts represents the possible options to CheckAttribute
type CheckAttributeOpts struct {
	CachedOnly    bool
	AllAttributes bool
	Attributes    []string
	Filenames     []string
}

func (repo *Repository) CheckAttribute(opts CheckAttributeOpts) (map[string]map[string]string, error) {
    ...
    return name2attribute2info, nil
}

The signature of this function isn't great but it could easily be improved.

@a1012112796
Copy link
Member Author

Why aren't you using check-attr and already written function that uses it?

I know it. But if use this cmd, should create tmp repo(this cmd can't be used in a bare repo, example:
image
), which is a more big work. so I think just analys this file content directly is more simple.

modules/git/repo_attribute.go Outdated Show resolved Hide resolved
@lunny
Copy link
Member

lunny commented Mar 1, 2021

Why aren't you using check-attr and already written function that uses it?

I know it. But if use this cmd, should create tmp repo(this cmd can't be used in a bare repo, example:
image
), which is a more big work. so I think just analys this file content directly is more simple.

So the above function CheckAttribute does not work?

@lunny lunny added this to the 1.15.0 milestone Mar 1, 2021
@zeripath
Copy link
Contributor

zeripath commented Mar 1, 2021

Ah... Ok. There's a way to make it work in bare repos - you read-tree into a (temporary) index file and then use that index file with --cached. (Which reminds me as to why I hadn't done this already.)

I guess we need to decide which is better - potentially parsing the file incorrectly ourselves versus the complexity of the above operation. Certainly if we're planning on calling gitattribites a lot we don't want to be writing and throwing away index files all the time.

I'll have to take a look at the .gitattribites format - iirc it's kinda complex with lots of globbing. We'll have to be cautious with reading this.

@a1012112796
Copy link
Member Author

Ah... Ok. There's a way to make it work in bare repos - you read-tree into a (temporary) index file and then use that index file with --cached. (Which reminds me as to why I hadn't done this already.)

I see, but maybe create index file in the bare repo is a dangerous work, becaue maybe will be double write ...
so maybe the most simple way is create tmp repo, and do check-attr in it...

@zeripath
Copy link
Contributor

zeripath commented Mar 2, 2021

No, you can create an index file anywhere and make git use it with the environment variable GIT_INDEX_FILE. Creating a temp repo is very slow.

@zeripath
Copy link
Contributor

zeripath commented Mar 2, 2021

Ok GIT_INDEX_FILE=$TMP_INDEX_FILE git restore --staged $DEFAULT_BRANCH -- .gitattributes can be used to set a single blob as an index then GIT_INDEX_FILE=$TMP_INDEX_FILE git check_attr --cached could be used to query it.

The reason for the restore rather than read-tree is that read-tree has to read the entire tree and subtrees so restore should be quicker.

It would be great if we could figure out a way to tell check-attr to just use a piped in .gitattributes but I don't think that's possible.

@a1012112796
Copy link
Member Author

Ok GIT_INDEX_FILE=$TMP_INDEX_FILE git restore --staged $DEFAULT_BRANCH -- .gitattributes can be used to set a single blob as an index then GIT_INDEX_FILE=$TMP_INDEX_FILE git check_attr --cached could be used to query it.

The reason for the restore rather than read-tree is that read-tree has to read the entire tree and subtrees so restore should be quicker.

It would be great if we could figure out a way to tell check-attr to just use a piped in .gitattributes but I don't think that's possible.

I see, but sadly can't use restore cmd in here. firstly, the restore can't be used in bare repo (It has NEED_WORK_TREE label) . secondly, it fact. .gitattributes can be added in subtree also.
now I has changed to use read-tree and check-attr. But I wonder whether it's a better solution than previous, because the check-attr can check only one file in one time ....

return "", false
}

name2attribute2info, err := gitRepo.CheckAttribute(git.CheckAttributeOpts{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK Now I understand what this is doing - I think we're gonna need a pipe which goes to write to an open

git check-attr --stdin ...

and then read and parse the line produced per path.

Otherwise you're going to get n calls to git check-attr per repository.

@zeripath zeripath modified the milestones: 1.15.0, 1.16.0 Jun 23, 2021
zeripath added a commit that referenced this pull request Sep 9, 2021
…guage stats and diffs (#16773)

Replaces #16262
Replaces #16250
Replaces #14833

This PR first implements a `git check-attr` pipe reader - using `git check-attr --stdin -z --cached` - taking account of the change in the output format in git 1.8.5 and creates a helper function to read a tree into a temporary index file for that pipe reader.

It then wires this in to the language stats helper and into the git diff generation.

Files which are marked generated will be folded by default.

Fixes #14786
Fixes #12653
@a1012112796
Copy link
Member Author

Close via #16773

@a1012112796 a1012112796 deleted the attr-lang branch September 10, 2021 04:34
@a1012112796 a1012112796 removed this from the 1.16.0 milestone Sep 10, 2021
@go-gitea go-gitea locked and limited conversation to collaborators Oct 19, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. type/enhancement An improvement of existing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request]: Ignore directories from language statistics bar
5 participants