This content originally appeared on DEV Community and was authored by Tai Kedzierski
I was trying to understand how some syntax in Groovy was working and it led me to some better udnerstanding on how closures can be used. I'm sure it's all in the documentation but... well I'm the kind of person wh really needs to see it in place to understand...
List and map "comprehensions"
Borrowing from Python, I was trying to understand how to perform "list comprehension" and "dict comprehension", but in Groovy. An article I found did give me the answer, and I noticed something a a little odd:
// List comprehension to double the values of each item
my_list = my_list.collect({ item ->
item * 2 // the last statement of the block is the item that gets "returned" for each item
})
// Map comprehension to add ".txt" suffix to each entry
my_map = my_map.inject([:]) { new_map, item ->
new_map[item.getKey()] = "${item.getValue()}.txt"
new_map // the returnable must be the final statement
}
(for details, do see the original post)
Two things that I noticed:
- In the list comprehension, the curly block is a direct argument to
.collect()- but in the map comprehension, the curly block is placed after the call to.inject() - In both cases, the block looks uspiciously like the
.each { name -> operations; }notation
So I did what any system-dismantling child would do... and tried something ...
def myfunc(message, thing) { println thing }
myfunc("Hello") { println "other action" }
This showed me that thing did actually populate.... with a closure !
basic_closure$_run_closure1@76f2bbc1
And thus my journey down to Wonderland bgan ...
.each { } is not a true loop
Notationally, it looks like using the each { } operation on an iterable item results in a loop that steps through each item.
Well, not really: the each() function (which, yes, it is) in fact takes one argument: a function handler, and calls it once for each item in the iterable it operates from.
This is why this will not work:
def error_on_null(my_sqeuence) {
my_sequence.each { item ->
if(item == null)
break // Groovy will tell you that "break" must be used in a loop!
}
}
If I try to write the above in Python, this is what is actually happening:
# For context - assume a Sequence type
# that implements the "each" function, like:
#class Sequence(list):
# def each(self, operation):
# for v in self: # self, the `list`
# operation(v)
def error_on_null(my_sequence):
def __is_null(item):
if item is None:
break # we are not directly in a loop in this scope
my_sequence.each(__is_null)
We can see more explicitly in this way what is happening: the content that checks null-ness is actually in a block and scope of its own - it does not incorporate its own loop, and so it is a syntactic error to try to use break there.
When using curly braces for a code block, we are actually defining an anonymous function, and passing it along to the each() function which itself implements the loop. This anonymous function is what is known in Groovy as a closure, a piece of code declared in one scope, and executed anywhere else, probably at a deferred time.
Similarly, with .collect( {} ) we are passing a closure, that can then be called by .collect's internal logic.
Closure parameters
Closures can have parameters too.
def greet = { greeting, name ->
println "$greeting , $name"
}
greet("Hello", "Tam")
And that's how you get the name -> notation in the .each { } call we are so much more familiar with.
Domain Specific Language: Jenkinsfile
I always did wonder how Jenkinsfile pipelines declared its own code blocks like
stage("Build stuff") {
sh "make clean && make && make install"
}
It turns out, the stage() function is defined something like this
def stage(stage_name, operation) {
org.hudson.etc.setStageName(stage_name) // for example
operation()
}
When stage() is called in my Jenkinsfile, it receives the closure I supply after it as an operation to perform. My closure (the build steps) has access to the variables and namespace in the rest of the file - and so can be passed along to the Jenkins-level stage() function which proceeds then to calling it (probably wrapped around some more complex error-handling logic).
The Closure Gotcha
Previously I posted about a behaviour I did not understand where variables were seeingly interpolated at the very last possible moment. The reason was, of course, because of closures !
I managed to replicate the issue with the following snippet:
// Mock Jenkinsfile directives
def string(opts) {
def name = opts.get("name")
def value = opts.get("value")
return (String) "${name} from ${value}"
}
def nodesByLabel(label) {
return ["agent-001", "agent-002", "agent-003"]
}
def build(opts) {
def job = opts.get("job")
def parameters = opts.get("parameters")
println "Building $job with params $parameters"
}
def parallel(ops_map) {
for(operation in ops_map) {
def op = operation.getValue()
op()
}
}
// --- My script
prepared_node_tests = [:]
def prepare_all_nodes_for_test(agent_label) {
def nodelist = nodesByLabel(label: "${agent_label}")
for (agentName in nodelist) {
println "Preparing task for " + agentName
prepared_node_tests[ agentName ] = { // THIS IS A CLOSURE
build job: 'Single_build_job',
parameters: [
string(name: 'TEST_AGENT_LABEL', value: agentName),
]
}
}
}
prepare_all_nodes_for_test("custom_label")
parallel prepared_node_tests
The output:
Preparing task for agent-001
Preparing task for agent-002
Preparing task for agent-003
Building Single_build_job with params [TEST_AGENT_LABEL from agent-003]
Building Single_build_job with params [TEST_AGENT_LABEL from agent-003]
Building Single_build_job with params [TEST_AGENT_LABEL from agent-003]
The reason is that the closure evaluates at a deferred time, with knowledge of agentName at the time of execution - which is after the loop has completed, and so the value at execution time ends up being its last value from the loop in every case ...!
In my solution in my complaint post, I moved the closure out a function, which resulted in it taking the value with which the function was called - which remains constant for each call, and is not affected by the loop.
Finally, a mystery solved!
More gotchas
A previous gotcha in the form of GStrings also threw me for a while.
Another item I found whilst trawling for answers recently is how "glabal" variables work in a Groovy file script, and how code not encapsulated in a function relates to variables equally not encapsulated. In essence:
implicit_property = "Top level"
def implicit_local = "Not available inside functions"
println "Top level"
println implicit_property
println implicit_local
def a_func() {
println "In function"
println implicit_property // OK
println implicit_local // Fail - it is "local" to the "main()"
}
Groovy compiles to Java and so the above actually ends up looking like this:
// assume "println" is a thing
class my_file_name {
static String implicit_property = "Top level"
public static void main(String[] args) {
String implicit_local = "Not available inside functions"
println "Top level"
println implicit_property
println implicit_local
}
public void a_func() {
println "In function"
println implicit_property // OK
println implicit_local // Fail - it is "local" to the "main()"
}
}
.... see what happened...? 😱
Conclusion
A dive down this rabbit-hole finally allowed me to make sense of a core part of the Groovy language , and identify a very interesting gotcha.
This feels more of a symptom of closures being very implicit in Groovy - and possibly of the false similarities in notation beteen function declarations, and objects - that closure in my Jenkinsfile looked to me like an "object" in JavaScript (think JSON notation) and as such I had not expected it to be the source of the deferred execution.
Another day, another lesson.
This content originally appeared on DEV Community and was authored by Tai Kedzierski
Tai Kedzierski | Sciencx (2022-05-12T10:54:08+00:00) Groovy Gotchas – Loops, Closures, and Jenkins DSLs. Retrieved from https://www.scien.cx/2022/05/12/groovy-gotchas-loops-closures-and-jenkins-dsls/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.