Should PowerAsserts be more verbose? (repost from user@groovy)

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Should PowerAsserts be more verbose? (repost from user@groovy)

tibokruse
Hi all,

I asked this yesterday on the user@groovy mailing list, but since Spock users would be affected by this mostly (I believe Spock uses Groovy power asserts since they moved into groovy), I thought it might be wise to also ask here.

I am wondering whether the output of PowerAsserts should be increased
to help understand test failures more easily.
Here are some examples (column alignment will look bad with
non-monospace fonts):

groovy:000> def id(it) {it}
===> true

groovy:000> assert [false, false] == id(['false', 'false'])
assert [false, false] == id(['false', 'false'])
                      |  |
                      |  [false, false]
                      false

groovy:000> assert [1, 2, 3, 4] == id('1'..'4')
assert [1, 2, 3, 4] == id('1'..'4')
                    |  |
                    |  [1, 2, 3, 4]
                    false

groovy:000> assert [1, 2, 3, 4] == new HashSet([1, 2, 3, 4])
assert [1, 2, 3, 4] == new HashSet([1, 2, 3, 4])
                    |  |
                    |  [1, 2, 3, 4]
                    false

groovy:000> assert [1, 2, 3, 4] == id('[1, 2, 3, 4]')
assert [1, 2, 3, 4] == id('[1, 2, 3, 4]')
                    |  |
                    |  [1, 2, 3, 4]
                    false

groovy:000> assert [1, 2, 3, 4] == id([1, 2, '3, 4'])
assert [1, 2, 3, 4] == id([1, 2, '3, 4'])
                    |  |
                    |  [1, 2, 3, 4]
                    false

As you can see the output in these cases does not help the user
understand what went wrong (assume the right hand side of the == being
something like foo()). This could be helped e.g. by adding hyphens
around Strings.

Another problem is checking escape symbols:

groovy:000> assert '\\\' \\\'' == id('\\\'\\\' \'')
assert '\\\' \\\'' == id('\\\'\\\' \'')
                   |  |
                   |  \'\' '
                   false

Here the power assert output is the final form of the string, not the
code-form. And it is a pain to mentally (un)escape to compare and find
differences.

So I would suggest extending the output such that:
- String values generally get quoted (with escaped special symbols),
- non-List collections get Type information added like "[x, y] as Set"
- Ranges are printed in code notation (e.g. 1..4 instead of [1, 2, 3, 4])

One of the small problems is that Strings like
'one\ntwo\rthree\r\nfour', which are rendered as multiline values,
will be rendered using \n, looking a bit uglier maybe.

Would that change be helpful? Can the behavior be changed without
breaking user code unduly?
What do you think?

Should I post updates about this here as well?

--
You received this message because you are subscribed to the Google Groups "Spock Framework - User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/spockframework.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Should PowerAsserts be more verbose? (repost from user@groovy)

Kostis Kapelonis
Hello

I find your examples very interesting. However I think that Spock has
more targeted power asserts because they are the last step in unit
tests.
For example if I am writing a unit test where I expect a certain
HashSet in the end, then I find it hard to believe that I will pass as
expected result something that is not a hashset.
(I refer to your third example)

Also I think that Spock already does something extra in some cases.

Pure Groovy

assert "now" == "now there"
             |
             false

Spock

"now" == "now there"
         |
         false
         6 differences (33% similarity)
         now(------)
         now( there)

I personally have never found cases where I wished for more
information in Spock tests (others might have a different opinion)
Would it be possible to perform your changes only for failed tests? If
you could modify just the output when a power assert fails then no
user code will be broken.
Spock tests that already pass will continue to pass, while Spock tests
that fail will just gain more information.

Kostis

On Sat, Aug 22, 2015 at 1:42 PM,  <[hidden email]> wrote:

> Hi all,
>
> I asked this yesterday on the user@groovy mailing list, but since Spock
> users would be affected by this mostly (I believe Spock uses Groovy power
> asserts since they moved into groovy), I thought it might be wise to also
> ask here.
>
> I am wondering whether the output of PowerAsserts should be increased
> to help understand test failures more easily.
> Here are some examples (column alignment will look bad with
> non-monospace fonts):
>
> groovy:000> def id(it) {it}
> ===> true
>
> groovy:000> assert [false, false] == id(['false', 'false'])
> assert [false, false] == id(['false', 'false'])
>                       |  |
>                       |  [false, false]
>                       false
>
> groovy:000> assert [1, 2, 3, 4] == id('1'..'4')
> assert [1, 2, 3, 4] == id('1'..'4')
>                     |  |
>                     |  [1, 2, 3, 4]
>                     false
>
> groovy:000> assert [1, 2, 3, 4] == new HashSet([1, 2, 3, 4])
> assert [1, 2, 3, 4] == new HashSet([1, 2, 3, 4])
>                     |  |
>                     |  [1, 2, 3, 4]
>                     false
>
> groovy:000> assert [1, 2, 3, 4] == id('[1, 2, 3, 4]')
> assert [1, 2, 3, 4] == id('[1, 2, 3, 4]')
>                     |  |
>                     |  [1, 2, 3, 4]
>                     false
>
> groovy:000> assert [1, 2, 3, 4] == id([1, 2, '3, 4'])
> assert [1, 2, 3, 4] == id([1, 2, '3, 4'])
>                     |  |
>                     |  [1, 2, 3, 4]
>                     false
>
> As you can see the output in these cases does not help the user
> understand what went wrong (assume the right hand side of the == being
> something like foo()). This could be helped e.g. by adding hyphens
> around Strings.
>
> Another problem is checking escape symbols:
>
> groovy:000> assert '\\\' \\\'' == id('\\\'\\\' \'')
> assert '\\\' \\\'' == id('\\\'\\\' \'')
>                    |  |
>                    |  \'\' '
>                    false
>
> Here the power assert output is the final form of the string, not the
> code-form. And it is a pain to mentally (un)escape to compare and find
> differences.
>
> So I would suggest extending the output such that:
> - String values generally get quoted (with escaped special symbols),
> - non-List collections get Type information added like "[x, y] as Set"
> - Ranges are printed in code notation (e.g. 1..4 instead of [1, 2, 3, 4])
>
> One of the small problems is that Strings like
> 'one\ntwo\rthree\r\nfour', which are rendered as multiline values,
> will be rendered using \n, looking a bit uglier maybe.
>
> Would that change be helpful? Can the behavior be changed without
> breaking user code unduly?
> What do you think?
>
> Should I post updates about this here as well?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Spock Framework - User" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [hidden email].
> To post to this group, send email to [hidden email].
> Visit this group at http://groups.google.com/group/spockframework.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Spock Framework - User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/spockframework.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Should PowerAsserts be more verbose? (repost from user@groovy)

tibokruse
Interesting, I had no idea Spock had extended the asserts further. So if Groovy changes, I guess Spock will not be immediately be affected, but Spock could change in the similar way.

The code change for Spock then would really be just calling another method to print the values.

And of course, the extended value would only change anything for failed tests.



On Monday, August 24, 2015 at 9:07:31 PM UTC+2, Kostis Kapelonis wrote:
Hello

I find your examples very interesting. However I think that Spock has
more targeted power asserts because they are the last step in unit
tests.
For example if I am writing a unit test where I expect a certain
HashSet in the end, then I find it hard to believe that I will pass as
expected result something that is not a hashset.
(I refer to your third example)

Also I think that Spock already does something extra in some cases.

Pure Groovy

assert "now" == "now there"
             |
             false

Spock

"now" == "now there"
         |
         false
         6 differences (33% similarity)
         now(------)
         now( there)

I personally have never found cases where I wished for more
information in Spock tests (others might have a different opinion)
Would it be possible to perform your changes only for failed tests? If
you could modify just the output when a power assert fails then no
user code will be broken.
Spock tests that already pass will continue to pass, while Spock tests
that fail will just gain more information.

Kostis

On Sat, Aug 22, 2015 at 1:42 PM,  <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="25_wn9Z-BQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">tibo...@...> wrote:

> Hi all,
>
> I asked this yesterday on the user@groovy mailing list, but since Spock
> users would be affected by this mostly (I believe Spock uses Groovy power
> asserts since they moved into groovy), I thought it might be wise to also
> ask here.
>
> I am wondering whether the output of PowerAsserts should be increased
> to help understand test failures more easily.
> Here are some examples (column alignment will look bad with
> non-monospace fonts):
>
> groovy:000> def id(it) {it}
> ===> true
>
> groovy:000> assert [false, false] == id(['false', 'false'])
> assert [false, false] == id(['false', 'false'])
>                       |  |
>                       |  [false, false]
>                       false
>
> groovy:000> assert [1, 2, 3, 4] == id('1'..'4')
> assert [1, 2, 3, 4] == id('1'..'4')
>                     |  |
>                     |  [1, 2, 3, 4]
>                     false
>
> groovy:000> assert [1, 2, 3, 4] == new HashSet([1, 2, 3, 4])
> assert [1, 2, 3, 4] == new HashSet([1, 2, 3, 4])
>                     |  |
>                     |  [1, 2, 3, 4]
>                     false
>
> groovy:000> assert [1, 2, 3, 4] == id('[1, 2, 3, 4]')
> assert [1, 2, 3, 4] == id('[1, 2, 3, 4]')
>                     |  |
>                     |  [1, 2, 3, 4]
>                     false
>
> groovy:000> assert [1, 2, 3, 4] == id([1, 2, '3, 4'])
> assert [1, 2, 3, 4] == id([1, 2, '3, 4'])
>                     |  |
>                     |  [1, 2, 3, 4]
>                     false
>
> As you can see the output in these cases does not help the user
> understand what went wrong (assume the right hand side of the == being
> something like foo()). This could be helped e.g. by adding hyphens
> around Strings.
>
> Another problem is checking escape symbols:
>
> groovy:000> assert '\\\' \\\'' == id('\\\'\\\' \'')
> assert '\\\' \\\'' == id('\\\'\\\' \'')
>                    |  |
>                    |  \'\' '
>                    false
>
> Here the power assert output is the final form of the string, not the
> code-form. And it is a pain to mentally (un)escape to compare and find
> differences.
>
> So I would suggest extending the output such that:
> - String values generally get quoted (with escaped special symbols),
> - non-List collections get Type information added like "[x, y] as Set"
> - Ranges are printed in code notation (e.g. 1..4 instead of [1, 2, 3, 4])
>
> One of the small problems is that Strings like
> 'one\ntwo\rthree\r\nfour', which are rendered as multiline values,
> will be rendered using \n, looking a bit uglier maybe.
>
> Would that change be helpful? Can the behavior be changed without
> breaking user code unduly?
> What do you think?
>
> Should I post updates about this here as well?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Spock Framework - User" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="25_wn9Z-BQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">spockframewor...@googlegroups.com.
> To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="25_wn9Z-BQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">spockfr...@googlegroups.com.
> Visit this group at <a href="http://groups.google.com/group/spockframework" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://groups.google.com/group/spockframework&#39;;return true;" onclick="this.href=&#39;http://groups.google.com/group/spockframework&#39;;return true;">http://groups.google.com/group/spockframework.
> For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Spock Framework - User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/spockframework.
For more options, visit https://groups.google.com/d/optout.