Sunday, August 23, 2015

Scythebill 12.4.1 - some small fixes

August 24 update: there was a glitch in Scythebill 12.4.1 that affected users that hadn't already upgraded to Scythebill 12.4.0 that showed an error screen right at startup.  I've released Scythebill 12.4.2 with a fix for that problem.  As always, please let me know if you have any problems, and thanks to the users who quickly reported this problem.

(Original post follows)

Since Scythebill 12.4.0 was released, I've had a few bug reports adding up to enough to justify a new release.  For most of you, if you've already downloaded 12.4.0, there's no particular reason to upgrade immediately (except, perhaps, on MacOS).

Here's what's been fixed in 12.4.1, from most important to least:

  • MacOS only:  the names of files saved from Scythebill were missing file extensions (".csv", ".xls", etc.).
  • Scythebill 12.4.0 renamed the country of "Burma" to its official name, Myanmar.  If you had manually entered (or imported) Myanmar as a country, upgrading to 12.4.0 could result in having corrupted location data.  No sightings were lost, but location names could be lost.
  • Importing eBird checklists with very long names (anything over 5 words) would fail to find existing locations with matching names.
  • Avisys and Birdbase imports with note data containing a backslash ("\") could fail.
  • Finally, there's a few small checklist improvements.

The one new feature is a small improvement to Avisys imports.  Scythebill will now correctly detect any use of the heard-only, male, female, immature, or photographed flags in Avisys exports.

Huge thanks to all those that reported bug!  As always, download here, and let me know if you have any problems, either on Google+Facebook, or by email

Wednesday, August 19, 2015

Scythebill 12.4 - BirdBase imports and the new eBird/Clements taxonomy!

Scythebill 12.4 is here! As always, download here, and let me know if you have any problems, either on Google+Facebook, or by email.  There's two highlights this time around:

  • The eBird/Clements 2015 taxonomy is now available
  • You can import directly from BirdBase exports!

eBird/Clements 2015 is now available

Scythebill 12.4 includes the just released eBird/Clements 2015 taxonomy, with a mass of splits, like a 3-way split of Asian Paradise-Flycatcher, a 6-way split of Purple Swamphen, and a 15-way split of Variable Dwarf-Kingfisher!  There's also a few new undescribed forms, like "New Caledonian" Storm Petrel, "Bananal" Spinetail, and "Elgin" Buzzard.

I've updated all the checklists to account for these splits, but I'd caution that some of the work here is necessarily tentative.  There is fairly little reliable information on some of these splits - if anyone reading this has solid data on the distribution of buzzards in southeast Asia, I'd certainly love to hear it!

All users - whether they use the eBird/Clements checklist or the IOC checklist - may have a bit of work to do when they upgrade.  Scythebill does all it can, using the subspecies you've entered and its checklists, to minimize the work, and the process is described in detail here in the manual.  And you can always punt that work to later if there's splits you're not ready to resolve.

Birdbase imports!

BirdBase recently closed its doors after literally decades of supporting the birding community.  For all those users, Scythebill now supports directly importing BirdBase CSV exports!  Just visit the Import screen, click the new Import from Birdbase... button, and open up a BirdBase export, and Scythebill will have all your data imported in a flash.

The Scythebill manual will (shortly) be updated to give some caveats and more details about the details of Birdbase imports, but generally it should work fairly simply.  A few details that might be interesting here:
  • The "S1" through "S4" custom flags will, when set, be added to your sighting notes. You can use Show reports to find all notes containing those strings and then bulk edit to set a Scythebill field like "Heard only".
  • Scythebill will add one of "BirdBase Home", "BirdBase Region", or "BirdBase Local" locations when those BirdBase flags are set.
  • BirdBase did not include a complete and modern set of countries, so a few liberties have been taken (all "Yugoslavia" sightings are placed in Serbia;  all Leeward Islands sightings end up in Montserrat; and so forth).
As always, after you import, I'd recommend visiting "Verify against checklists..." to look for issues revealed during the import (in particular, any mistakes Scythebill made assigning sightings to species - it's generally pretty darn good, but not perfect).

And if you have any problems with BirdBase imports, please let me know.  I'm more than happy to personally import your file to Scythebill so I can fix any problems that lurk in this new feature.

I also think that I've improved Avisys imports - I suspect that locations will import more cleanly, and that files that did not import well (and took a long time to even report failure) will now import more cleanly.

Other changes

The last version of Scythebill added preferences to control what you consider a lifer - skipping introduced or heard-only birds.  But I missed several places in Scythebill where lifers are enumerated or listed (for example, in the "only lifers" view of a checklist).  These should all be working as expected now.

The Canary Islands were a bit confused in Scythebill - available both in the Atlantic Ocean and as the Canarias province in Spain.  There's now just a single Canary Islands location, in the Atlantic Ocean (and the country code is fixed, so importing to eBird will work).

There's also the usual set of checklist improvements, though this time it's only a few dozen fixes.  As always, thanks to the users reporting problems.

The Browse by location screen would sometimes show "introduced" twice on a single line.  That's fixed.

Printing was failing for Firefox users;  this should now work.

Monday, July 27, 2015

Scythebill 12.3 is here!

Scythebill 12.3 is here! As always, download here, and let me know if you have any problems, either on Google+, Facebook, or by email.  There's two main highlights this time around:

  • The IOC 5.3 checklist - released 4 days ago - is now available.
  • Checklists have been massively improved, and are now globally cleaned up
There's also a few small features added this time around, most importantly better support for those of you who don't "count" introduced or heard-only species.

IOC 5.3 checklist

The IOC 5.3 checklist was released on July 22nd, and Scythebill's already up-to-date.  For the world listers out there, the highlights will likely be:
  • Purple Swamphen is split 5 ways.
  • Silver and Red-billed Gulls are lumped.
  • Four-colored and Gorgeous Bushshrikes are lumped.
... but for the full list, see the IOC website's species updates.

For compatibility with this update, I've proactively made two changes to the eBird/Clements taxonomy:
  • Added Sichuan Bush-Warbler as a full species
  • Added crassirostris as a subspecies of Blue-winged Parrotlet (which IOC splits as a full species - prematurely, perhaps)

Checklist improvements

Since the last release of Scythebill, I've been determined to finish cleaning up all the country checklists. And so I've gone through Europe, Asia, and the Pacific Ocean, correcting every last country's checklists against official checklists (or as close to official as I could get), adding and removing species, setting rarity status, and filling in earlier blanks for Andorra, San Marino, and South Sudan. The effort resulted in over ten thousand improvements!

Done, right? No! After that, I went through all 10,705 species of birds, eyeballing each checklist-built range map to see how it matches with an "official" range and to look for oddities.  That resulted in a few hundred more corrections and clarifications.

Once again, I'd strongly recommend that with this release you try out the "Verify against checklists..." option in the File menu.  I suspect that at this point most of its findings will point out mistakes in your existing sightings, but I'm sure there are still some glitches in the checklists.

There are also still areas where I could use your help.  I'm looking for anything resembling an official checklist for the following countries or regions (which have no checklists):
  • Kosovo
  • North Korea
  • Åland Islands
... and also for these countries, where the current checklists are largely guesswork based on surrounding countries:
  • Moldova
  • Macedonia
  • Bosnia
  • Albania
  • Jordan
  • Syria

Finally, I should note that the county-level checklists in Great Britain and Ireland are not in great shape.  I don't have access to resources that would let me clean up the existing ones or add new counties to complete the set.  I'm strongly considering removing these from the program altogether.  Let me know if you feel strongly one way or the other.

New (small) features and fixes

Getting the checklists cleared up took the vast majority of my time in this release, but I did get a chance to squeeze in a few improvements.

Scythebill's preference screen has two new checkboxes:

If you are the sort of lister that doesn't count heard-only sightings, or doesn't want to consider introduced birds, you've had two choices until now.  Either you didn't enter those sightings at all, or you hand-tuned reports to exclude those sightings.  Now, just click one checkbox and you're done!  Those sightings will still show up in reports, but they won't count in numeric totals, and they won't show up as "lifers" when you're entering species.

Scythebill will also now take advantage of the checklists to default a sighting's status to "introduced" or "not established".  I'd always wanted to do this, but didn't trust the checklist contents enough to do so.  Now I do!  Also, sightings of the feral form of Rock Pigeon will automatically be entered as "introduced".

Also, I've gotten a few requests to allow entry of approximate counts - ">20", "~50", and so forth.  So the count field now lets you enter any one character before the number.  (eBird doesn't support approximate numbers, so when Scythebill exports to eBird, it'll drop that character and just send the number.)

Finally, there's some small bug fixes:
  • Scythebill reported an error if you tried to bulk edit the date of sightings that didn't have any date.
  • Users sometimes got errors when adding new locations in Browse by location.

Sunday, May 17, 2015

Scythebill 12.2: checklist questions, lifer map, and more

Scythebill 12.2 is now available!  This release has a variety of smaller features, and some important bug fixes.  As always, download here, and let me know if you have any problems, either on Google+Facebook, or by email.  The news this time includes:
  • A quick and simple way to answer more checklist questions
  • An interactive map of country-by-country lifers around the world
  • Geographic cleanup of offshore islands for many countries, with a lot more "magic" locations to track full country lists.
  • Lots more checklist corrections and new checklists - all of the Americas are done!

Answer more checklist questions

The Scythebill checklists have made it easy to find out what lifers you can see in hundreds of areas of the world.  And you can always find out what would be new for your list right there - what Indian species would be new for your Indian list, for example.  But they haven't let you answer questions like:
  • What species in a US state would be new for an ABA list?
  • What rarities of the Canary Islands would be new for your Western Palearctic list?
  • What Australian birds would be new for your year list?
All these questions are easy to answer now!

Start with Browse by location, then:
  1. Find the checklist you want to examine.
  2. Change "Show everything" to New for a custom report...
  3. In the window, choose the report for which you'd like to find new species.  So if you wanted to find checklist entries that would be new for the Western Palearctic, choose "Location" "is in" "Western Palearctic", and click OK.
  4. That's it!  You'll now have a list of species that would be new for that report.
If you'd like to exclude rarities, just check the new Hide rarities? option.

Thanks to Jochen B. for the feature idea.

Interactive lifer maps

Scythebill's new Map of world lifers... menu item (in the File menu) brings up a map of the world with each country color-coded for how many lifers are waiting for you there.  Here's my map:

You can also zoom into a specific region of the world with a small menu in the upper left:

Geographic cleanup

Many countries around the world have offshore states or territories that do not fit into ordinary geographic bounds.  For example, the US state of Hawaii is in the Pacific Ocean, not North America, so that needs to be separated to keep our "North America" lists in proper form.  I've cleaned this up for a number of other countries:
  • Australia has "lost" a number of offshore territories.  The Ashmore and Cartier Islands and the Coral Sea Islands have independent entries.  (Norfolk and Macquarie Islands already had their own entries.)
  • The Colombian department of "San Andrés, Providencia, and Santa Catalina" has been moved to the West Indies.
  • South Africa's "Prince Edward and Marion Islands" have their own entry in the Indian Ocean.
  • The Netherlands Antilles have been divided in two.  The West Indies portion comprises  Sint Maarten, Sint Eustatius, and Saba.  The South American portion includes Bonaire and Curaçao.  (Aruba left the Netherlands Antilles in 1986, and is its own country in the Kingdom of the Netherlands).
  • Spain's "exclaves" of Ceuta and Melilla have been moved moved into Africa.
All of this is more geographically accurate, to be sure, but it made calculating a full country list much harder.  To fix this, I've added a slew of "magic" locations:
  • Australia (with dependent territories)
  • Chile (with offshore islands)
  • China (with administrative regions)
  • Colombia (with Caribbean islands)
  • United Kingdom (with dependent territories)
  • United States (with dependent territories)
  • Ecuador (with Galapagos)
  • France (with Overseas Territories)
  • New Zealand (with Tokelau)
  • Portugal (with Azores and Madeira)
  • Spain (with Canary Islands, Ceuta, and Melilla)
  • Norway (with Bouvet Island and Svalbard)
  • Finland (with Åland Islands)
  • Kingdom of Denmark
  • Kingdom of the Netherlands
  • South Africa (with Prince Edward and Marion Islands)
You can find the full definition of each of these in the online manual.

Checklist improvements

Lots of checklists have been cleaned up in this release.  In particular, I've finished manually verifying and setting rarity status checklists for all of North America (including Central America and the West Indies), making this the third completed continent (after South America and Africa);  this required corrections to:
  • All of the Canadian provinces and territories
  • Nicaragua, Honduras, El Salvador, Guatemala, Belize, and Greenland
(Whenever I tackle this, I tend to find some humorous mistakes.  This time the winner came from the El Salvador checklist, which was missing Great Blue Heron - and instead had Great-billed Heron, which would be an impressive range extension from southeast Asia!)

I've also reconciled the checklists (setting "rarity" status and adding/removing species as needed) for:
  • Portugal (removing birds recorded only on the Azores or Madeira)
  • Spain (removing birds found only recorded from the Canary Islands or Ceuta and Melilla)
  • France
  • Belgium
  • Greece
  • Australia
  • New Zealand
  • Socotra Island
  • Seychelles
  • Netherlands Antilles
  • Faroe Islands
  • Madeira
  • Azores
And there are brand new checklists for:
  • Heard Island and MacDonald Islands
  • Macquarie Island
  • Easter Island
  • Juan Fernandez Island (and its endemics removed from Chile)
  • Curaçao
  • Bonaire
  • Niue
  • Svalbard and Jan Mayen
  • Saint Pierre et Miquelon
  • Coral Sea Islands
  • Ashmore and Cartier Islands
There's additional, smaller fixes in southern Africa, the Indian Ocean, Costa Rica, and Peru.

Fixed bugs

I've fixed a couple of problems with the photos code added in the last release.
  1. If a photo was moved or deleted, and Scythebill couldn't find it, you'd get an unfriendly error message.  Now, you can relocate the photo and fix the problem.
  2. Under some circumstances, Scythebill might not save added photos.

Friday, April 24, 2015

Scythebill 12.1 - attach photos, IOC 5.2, better checklists, and more

Scythebill 12.1 is here!  As always, download here, and please let me know if you have any problems, let me know either on Google+Facebook, or by email.  The big features this time  include:
  • Attach photos - either from files or the web - to your sightings!
  • IOC 5.2 is available the very same day it's released!
  • Many, many improved checklists (including an entire continent!)
  • Some important "geocoding" and eBird bugs are fixed
Also, if you missed the announcement, Scythebill now has a real website!  Please visit and take a look.  Relatedly, the open-source repository has moved too, and is now available on Bitbucket, so if you'd like to follow Scythebill's development, that's where to go.

Photo attachments

Scythebill now lets you attach photos - both files and webpages - to your sightings.  This has been requested for a long time, and it's finally available.  The implementation here is a starting point, not a final destination, and I'm very curious what features you'd like to see added on top of this - please let me know!

When you're entering species, there's a new Photos column:

You can click on that camera icon - which will bring up a file chooser - or you can drag a file or a webpage onto that icon.  (Note: webpages don't seem to work in Linux.  I'll see if I can get this working in the near future.)  When you do either, that camera icon will change to a number;  and if you expand the sighting, you'll see the list of photos on the right side:

You can click on the photo name to open it - it'll open (outside of Scythebill) in whatever application typically displays photos or webpages on your computer.  You can also drag onto or click the "Drag a photo or click..." part of the page here to add more photos, or click the red X to remove a photo.

The same controls for adding and removing photos here are also everywhere you can edit sightings in Scythebill.

IOC 5.2 taxonomy

The IOC 5.2 taxonomy was just released today, and Scythebill's already up-to-date!  You can find the list of species-level changes here.  As always, the checklists have already been updated (though I admit I'm guessing a bit at the wintering range of Chinese vs. European Blackbird in southeast Asia).

I've also upgraded the IUCN Redlist status for all species (in both IOC and eBird/Clements) from 2012 to 2014.  (It will surprise no one that most changes are not for the better, though there's the occasional bright spot.)

Updated checklists - all of Africa!

IOC 5.2 came out a little bit later than I imagined, and I took advantage of this time by cleaning up all of the African checklists, including Atlantic and Indian Ocean islands near Africa.  They're definitely vastly better - with some embarrassing mistakes cleaned up (Broad-tailed Grassbird in Africa?!?) - but it's possible some errors have been introduced.  If you've travelled in Africa, please use Verify against checklists... in the File menu and let me know what you find.  (This might find mistakes in the checklists, or it might find mistakes in your data;  either way, I'm happy to hear.)

There's also brand-new checklists for:
  • Norfolk Island (with its endemics removed from the Australia checklist)
  • Mayotte (with its endemics removed from the Comoros checklist)
  • East Timor (also known as Timor-Leste), Nauru, and Clipperton Island
Finally, there've been some smaller fixes to checklists in East Asia and Great Britain.

Bugs fixed

Linux users should have a somewhat easier time of things with fonts.  Also, both Linux installs now come fully prepackaged, with no need for an extra Java install.

A few bugs were fixed with the "geocoding" support added in Scythebill 12.0 - automatically grabbing lat/long and county names from Google and eBird.  Most of the combinations are obscure - but particular thanks to Sara M. for finding and reporting the most conspicuous of them.

Three bugs were fixed when importing from eBird:
  • Some eBird exports are produced with 2 year dates - "1/1/15" instead of "1/1/2015". Scythebill parsed those as the year 15 - yes, just 15 - which is probably not what you wanted. That's fixed, and Scythebill will automatically clean up your sightings as soon as they're loaded if you've encountered this bug.
  • If eBird exported time in AM/PM format, Scythebill completely failed to parse any time starting with "12:".
  • Scythebill failed altogether to load eBird imports with some stranger "sp." combinations (in particular Merganser sp. - Mergus/Mergellus/Lophodytes sp.).

Saturday, March 7, 2015

Scythebill 12.0 - better locations!

Scythebill 12 is here!  As always, download here, and please let me know if you have any problems, let me know either on Google+Facebook, or by email.  The big items this time around include:

  • Greatly improved location support
  • 24 newly reconciled checklists - the United States and West Indies are (almost) done!
  • and Waarneming import support

Better locations

The big new feature is improved location entry.  Locations can now have latitude and longitude, and full text descriptions.  But since it's tedious to enter latitude and longitude, Scythebill will automatically search for those locations using Google and eBird's "hotspots"!  When it finds a result, it can:
  • Enter the latitude and longitude automatically
  • Show you a map of the location
  • Automatically find the state or even county, so you don't have to
Here's Keoladeo National Park in India - Scythebill (or, rather, Google) found it and automatically placed it inside Rajasthan State.

And here's Golden Gate Park, automatically located not just in the state of California but also into San Francisco County:

And, finally, Magoebaskloof in South Africa:

Latitudes and longitudes are now part of the Scythebill import and export format, and will also be grabbed out of eBird imports.

This is all new code, and there may be a few rough edges - please let me know if anything doesn't work.  (And in case you're wondering, no, Scythebill doesn't send any personal information - cookies, for example - to Google or eBird.)

More checklist improvements

A number of checklists have been reconciled for accuracy and rarity status:
  • The last of the states in the United States is done!  Kentucky was cleaned up by Bill D., and the wonderfully helpful Zack W. took care of Maine, Maryland, Hawaii, Florida, and the District of Colombia!
  • I've handled (almost all) of the West Indies - Antigua and Barbuda, the Bahamas, Bermuda, the British and US Virgin Islands, Cayman Island, Cuba... you get the point.  (The Netherland Antilles and (French) St. Martin aren't done yet;  information on these is harder to come by.)
  • Abdel B. contributed a number of corrections - mostly southeast Asia, but also some African fixes too. and Waarneming imports

Scythebill now offers some experimental support for importing from files - which also supports and  Do let me know if you have a chance to try this out!

Other fixes and improvements

  • Western Whistler - a species newly available in the last IOC update - can now be entered.  (I invented a new Clements subspecies of Golden Whistler for those who want to enter this in Clements to be ready for the future.)
  • The "Domestic Form" of Rock Pigeon now imports cleanly into eBird.
  • Reports now show a family count next to the species count.  Thanks to Eric S. for the idea!
  • If you use the arrow keys to navigate while entering species, a few species of context at the top and bottom always stay visible - again, thanks to Eric S. for the suggestion.
  • You can now edit the "Complete list of sightings?" setting for a visit after initial entry.
  • An error message that could appear when creating a new .bsxm file - especially for first time users - has been fixed.
  • If you clicked the "Back" button while entering sightings, you lost all your work and had to re-enter everything.  (Ugh.) It's been fixed.

Friday, January 23, 2015

Scythebill 11 - eBird, international names, and IOC 5.1

Scythebill 11 is here!   As always, download here, and please let me know if you have any problems, let me know either on Google+Facebook, or by email.

Scythebill 11 adds three big new features:
  • Much better eBird support - Scythebill can store start times, duration, distance, etc. - and rapidly get that data to eBird.
  • International names!  Tired of seeing bird names in English?  There's now 20 different languages available.  And if you'd rather use scientific names, that's possible too.
  • Last, but not least, the IOC 5.1 upgrade, announced last week.
There's lots of smaller improvements too!

(If you've been beta testing, the big change in this release is the IOC 5.1 upgrade;  the notes below describe everything that's changed for those that didn't use the betas.)

Full eBird Data

First, eBird is now much better supported.  Previously, you could only enter “Incidental” sightings, with rather little data.  Now you can choose a protocol, attach checklist comments, set start times, durations, distances, and the “Complete checklist?” field, and it'll all be included as you export to and import from eBird.  All of this extra data makes its way into the now-enhanced Scythebill export files too.

You can also get an eBird file saved off as soon as you finish entering sightings, sparing you a series of extra steps.

International bird names

Another big new feature is international bird names!  If you use the IOC checklist, you can get bird names in 20 different languages (plus English, of course).  Most of these are 90-95% complete, though a few (Portuguese, Estonian, and Hungarian) are a bit sparser.

You can also choose, if you wish, to tell Scythebill that you prefer scientific names over common names.

Just click on the new Preferences button on the main page to edit these settings.

IOC 5.1 - and undescribed species

The latest version of the IOC taxonomy is now available, and all checklists have been upgraded to follow its latests splits.  (The checklist ranges for the four new Fox Sparrow species are the best I could do with the data I had;  please let me know if you see anything amiss.)

I've also added - to both Clements and IOC - 26 different undescribed species.  (Two of these are, in fact, already described and have full names in the IOC checklist.)

Better editing of existing data

There’s now a much smoother experience editing data from your old outings.  Before, if you realized you’d made a mistake - missed some species, added some others incorrectly, forgot to enter a count, and so forth - you had to tediously find sightings one-by-one.  Now, just go back to Enter sightings, enter the date and location, click Next - and you’ll be right back where you were, with a full list of sightings from that visit.

Even better, there’s a menu on the Enter sightings page with your 10 most recently entered visits, so you can quickly make fixes to recently entered data:

Checklist improvements

These 5 countries now have fully reconciled checklists, including rarities:
  • Costa Rica
  • Iceland
  • Panama
  • St. Lucia
  • Germany (thanks to Jochen B.)

Zack W. did a huge job, reconciling checklists for 11 US states!  Michigan, New Jersey, New York, Oklahoma, Ohio, Rhode Island, Tennessee, Utah, Virginia, West Virginia, and Wisconsin all are up-to-date with rarities set.  And Bill D. reconciled the checklists for Georgia and Indiana - we're down to just five US states that need touching up.

  • The 25th ABA Checklist update is applied (e.g. Purple Swamphen is now an “introduced” species in Florida)
  • The Water/Brown-cheeked Rail split is now correctly reflected in checklists.
  • Daniel H. found an error in the Panama checklist
  • Jochen B. pointed out mistakes in a number of countries
  • Paul H. saw problems in the Western Australia and Norfolk lists.
  • Niall P. provided some South African corrections.
  • Johan L. identified an issue with the Trinidad and Tobago list.

Smaller things

The "Clements" checklist is now listed as the "eBird/Clements" checklist - it was renamed that a couple of years ago.

Several users found a (Mac-only?) bug - if you started Scythebill by opening a .bsxm file, instead of opening the Scythebill application directly, you saw an error message that your file was corrupt.  Files weren’t corrupt, and everything worked, but it was scary all the same.

In Show reports, when you create a date-based report “is after” and “is before” were wrong - if you asked for “is before 2010”, you got everything before 2010 and 2010 itself.  Now it does what it says.  Thanks to Steve C. for that report.

A nasty bug (Windows only) that could show an error dialog and lock up Scythebill altogether (you had to kill it with Task Manager) has been fixed (most likely - I couldn't personally reproduce it).  Thanks to Penny V. for reporting this.