Timeline for answer to Sandbox for Proposed Challenges by user45941
Current License: CC BY-SA 3.0
Post Revisions
19 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 1, 2016 at 13:27 | comment | added | Fatalize | The problem with this scoring technique is that even if people make legit attempts at counting the cats, they will try to optimize their code for the limited test set, which will make their approaches less general on new data. One solution could be to provide a validation set (for say 2 weeks if that's the time frame in which you can post an answer) on which people can evaluate their answers, and choose the winning answer based on a test set that was not available during those 2 weeks containing new images. | |
| Jun 30, 2016 at 4:34 | history | edited | user45941 | CC BY-SA 3.0 |
added 157 characters in body
|
| Jun 29, 2016 at 9:14 | comment | added | Fatalize | @Mego It probably would be easier because it wouldn't force people to use machine learning techniques (You could use texture comparison to distinguish feathers from scales, color detection, etc.). Counting the number of each is an added difficulty that makes it maybe too difficult as a whole. Only classifying is easier but probably a dupe of the goats challenge. Only counting might be good though. | |
| Jun 29, 2016 at 9:10 | comment | added | user45941 | @Fatalize Would using non-mammilian animals (such as lizards and birds) be better in your opinion? | |
| Jun 29, 2016 at 9:09 | comment | added | Fatalize | Distinguishing a cat from a dog on a picture already requires decently advanced algorithms (CNNs are the first that come to mind) which in turn require a LOT of training data to generalize decently. | |
| Jun 28, 2016 at 2:48 | comment | added | user45941 | @xnor On the other hand, without the distracters, it would likely be too easy to get a perfect score. | |
| Jun 27, 2016 at 23:48 | comment | added | xnor | I think the part about the dog makes this a chameleon challenge -- distinguishing a cat from a dog is much harder than identifying and counting cats on an image free of distracters. | |
| Jun 27, 2016 at 5:41 | comment | added | user45941 | @Downgoat As per usual with these types of challenges, optimizing for the test cases is not allowed. However, given that a ML-oriented approach would be interesting and perform well on this challenge, I'd be willing to include a training set, separate from the scoring set. As for limits on possible inputs, I'm trying to work out a good set. | |
| Jun 27, 2016 at 5:15 | comment | added | Downgoat | Is fitting allowed? I think learning will probably be the most practical approach to this challenge. Though this does need some specification before it's ready to post imo and limits on the possible inputs as I doubt any answer will be able to conform to the wide variety of things cats can be e.g. color, position, shape, size, direction, camera position, etc. | |
| Jun 27, 2016 at 1:47 | history | edited | user45941 | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Jun 27, 2016 at 1:46 | comment | added | user45941 | @Liam On the other hand, if all of the test images consisted of 0-5 cats and no other animals, it would be too easy to get a perfect score. Having harder test cases means that submissions will have room for improvement, and thus there will be more competition. | |
| Jun 25, 2016 at 22:45 | comment | added | Liam | Showing a picture of a dog and expecting a zero output doesn't seem fair. Writing a program that distinguishes between dogs and cats seems like a difficult enough task as it is, and you have it here as sort of an afterthought. Empty images should be emptier, imo. | |
| Jun 23, 2016 at 20:12 | comment | added | user45941 | @Upgoat The time requirement can be extended. The number of cats will always be clear to a human viewer with 20/20 eyesight and adequate attention for detail. The challenge is about counting cats, not about picking out camouflaged cats, so there won't be any excessive trickery in the test images. Built-ins are allowed, but I'm on the fence about whether or not I'm going to allow them to be competitive, since Mathematica is likely the only language that would benefit from built-ins. | |
| Jun 23, 2016 at 20:07 | comment | added | Downgoat | Also, I think there should be some specification for how the cats would appear. At least having the full cat's face showing would make this challenge more approachable. If a test-case shows the back of a cat that seems a bit too difficult for a PPCG challenge. Also: Will there be a mix of cats and other animals? Are built-ins allowed (I'm very sure there's mathematica builtin)? | |
| Jun 23, 2016 at 20:03 | comment | added | Downgoat | "Submissions must output and terminate within 10 seconds for a single image." I don't know about that, counting the amount of cats and only cats is a rather difficult task already, a 10-second limit seems really small. Perhaps a couple minutes? | |
| Jun 23, 2016 at 19:24 | history | edited | user45941 | CC BY-SA 3.0 |
added 15 characters in body
|
| Jun 23, 2016 at 19:19 | comment | added | user45941 | @FryAmTheEggman I don't think it's a dupe of goats - goats wanted a boolean classification, where this one asks for a count. The rice question is closer, but there is a possibility (and perhaps a need) for different approaches, given that a) cats can overlap, b) cats come in different colors, and c) cats have much more complex shapes than rice grains (which are ellipses). | |
| Jun 23, 2016 at 19:17 | comment | added | FryAmTheEggman | I feel like this is probably a dupe of either the goat question or the rice question. I'm not sure that it being a test battery question or the precise subimage to identify really makes it much different from these other ones? | |
| Jun 23, 2016 at 18:52 | history | answered | user45941 | CC BY-SA 3.0 |
