Repository navigation
regexpMatch() #6479
Description
Hi,
I'm using regexpMatch() while analyzing C code, but there's no matched result even the pattern is correct.
For example, in my C code:
MY_FUNC(ARG0, ARG1, NULL,
"\n%sDoor %s is open", arg4, arg5->arg6);
The query I created is:
import cpp
from FunctionCall fc, int i
where
fc.getArgument(i).getValue().regexpMatch("\n%sDoor %s is open")
select fc, "Function: "+fc.getTarget().getName()+" Log: "+fc.getArgument(i).getValue()
when I test on LGTM, "\n%sDoor %s is open" shows 'invalid', the errors are
missing ')' at '%'
missing 'select' at 'sis'
it's hard to tell how to fix the pattern.
I test the pattern on regex101.com, which has no issue.
So what regex language does codeql use? and how to write the correct query in this case.
Thanks in advance!
Activity
Marcono1234 commentedon Aug 13, 2021
Cannot reproduce; this query compiled without any issues. Do you maybe copied by accident some invisible characters which could be causing the errors?
CodeQL uses Java's
Patternsyntax; regex101 also supports that since a few months.The regex pattern you wrote looks fine to me, though it appears that it does not use any regex functionality, the
\nis already converted to a LF by the CodeQL string literal (see language specification). So instead of usingregexpMatch("...")you could just compare the string directly:= "..."Maybe the maintainers have an idea what could be causing this issue for you.
xiaoxinz-cisco commentedon Aug 16, 2021
Thanks, let me give you more details.
There's a line of code in sample project 'fish-shell/fish-shell':
b.append_format(_(L"(Type 'help %ls' for related documentation)\n"), cmd);How to write a regex to match string
"(Type 'help %ls' for related documentation)\n"?In regex101,
\(Type.*matches, but it doesn't work in codeqlfish-shell/fish-shell: https://lgtm.com/projects/g/fish-shell/fish-shell/?mode=list
RasmusWL commentedon Aug 16, 2021
Hi @xiaoxinz-cisco, you should use
\\(instead of\(since the fact that this string is sent to Java requires an other layer of regex-escaping -- do let me know if that solves your problem 😊For example, to reproduce the Python regex
'<(?:(?P<converter>[^>:]+):)?(?P<parameter>\w+)>'we had to write"<(?:(?<converter>[^>:]+):)?(?<parameter>\\w+)>"xiaoxinz-cisco commentedon Aug 16, 2021
Thanks @RasmusWL , what are the reasons for the below patterns' failures?
Marcono1234 commentedon Aug 17, 2021
Note that with Java's
Pattern.by default does not match line terminators. Because the string you want to match contains a LF (\n) it will therefore not match.You have to prefix the pattern with
(?s)to make.match line terminators as well:(Query console link)
However, if you only check for a prefix you could use CodeQL's
indexOf(ormatches) predicate as well:(Query console link)
RasmusWL commentedon Aug 17, 2021
Thanks @Marcono1234 for highlighting that part about Java's pattern and line terminators 👍
I would strongly recommend you use
matches("<...>%")instead ofindexOf("<...>") = 0, since it's a bit faster to evaluate thematches.xiaoxinz-cisco commentedon Aug 19, 2021
Thanks @RasmusWL @Marcono1234 , is the regex rules in codeql documented somewhere? such as the specific Java pattern and line terminators rule.
Marcono1234 commentedon Aug 19, 2021
If I understand it correctly, CodeQL is exactly using Java's
java.util.regex.Pattern, so everything described in its documentation applies to CodeQL as well.The CodeQL predicate documentations (for example for
regexpMatch) and the CodeQL language specification also refer to Java'sPattern, and this applies regardless of scanned programming language.RasmusWL commentedon Aug 23, 2021
Yep, at the bottom of the table here, it says:
github-actions commentedon Sep 7, 2021
This issue is stale because it has been open 14 days with no activity. Comment or remove the
Stalelabel in order to avoid having this issue closed in 7 days.github-actions commentedon Sep 14, 2021
This issue was closed because it has been inactive for 7 days.