One of the longest-serving computers I have kicking around is a shell server which since 1993(ish) has hosted my email, domain names, and websites.
Of course it’s been patched and reinstalled but I never bothered actually clearing the configurations up to remove old projects. Yesterday it suffered a bit of a meltdown and I took the initiative to finally do some tidying, a process which largely involved manually going through everything bit-by-bit, line-by-line.
I finally got sick of my Apple Airport Express network not quite behaving, and having to jump through hoops to play from the big fileserver holding my MP3 collection, and having to deal with sodding iTunes. My current workplace has a pair of Sonos units which are used in the office as a jukebox for the staff to play whatever music they want, and that seemed to tick most of my boxes.
My calendar has been a mess for a while – or rather, the organisation of calendars. For years I’ve had several devices and keeping them in synchronisation was a nightmare – from my original Palm Vx handheld right up to the iPad and iPhone I now use it’s been tricky, not helped by absence of decent CalDAV software or support for calendar servers on the devices I’ve used. Add into the mix almost 20 years of historic calendar data (where sometimes I do need to find the last time I saw a particular client for instance) and it’s a recipe for disaster.
When Apple released MobileMe it got a bit better. Around 2004ish I managed to export my calendar from the Palm software using Missing Sync and into iCal, and thus into MobileMe. Through that I frigged it to talk to my phone at the time, a Nokia E70 (I think). Then when iCloud came along I ported it and my iPad, iPhone, Mac and the web stuff was all integrated. Hoorah.
Then this week it all went tits-up. I decided to merge two work calendars so Nicky could share them, by exporting both calendars as ICS files and then importing them into a new calendar. Of course, connectivity at a client’s this week has been very flakey as they’re moving offices and iCloud seemed not to like this when it attempted to sync the new entries which manifested itself as the iCal client app giving me 403 errors (“UID already exists”). No amount of Googling and buggering about with Apple Support would fix it so I took matters into my own hands.
Problem 1 – Getting the calendar entries in without having to retype 20 years of historical data: the ICS ‘UID’ field on iCloud is unique to the group of calendars it seems, which are grouped under one username. The solution was to export the ICS file (without connectivity, so iCloud didn’t attempt to be clever) and run a small perl script which would append “JFIX-” to the UID and then re-import. This then gave me a new calendar with new entries, which I could then sync with iCloud (it took about an hour for it to stop ‘Updating…’). So far so good, but after an hour I discovered…
Problem 2 – events appearing in iCloud and iCal don’t propogate to iPad or iPhone: Turns out that UID needs to be hexadecimal for it to work properly, so I re-ran my perl script to append ‘ABCD-‘ instead. This createdanother set of UIDs which I could import into a new calendar. This worked fine, and has propagated to all my iDevices.
In Googling this by the way I found a series of touted solutions, one of which was to use grep to remove UID lines altogether. iCloud really hates this and goes on a duplication fest, adding in events sometimes up to 15 times (but most likely adding between 5 and 10). I also worked out after a while that the easiest way of finding out what’s really in iCloud is to look on the web client, as this is what gets sent to the phones, iPads, etc. If it’s not there, it’s not going to propagate or be visible.
Lesson learned though is: (a) be prepared for a nightmare if you want to en-masse move events from one calendar to another, and (b) make sure your connectivity is rock-solid and don’t be impatient.
You might have noticed over the years but I take a lot of photographs, and like many photographers I keep them in a catalogue on a computer: this catalogue allows me to find photos quickly and easily, and also apply retouching, colour changes, and all that sort of thing. The application I use is called Adobe Lightroom and I’ve been a user since version 2 back in 2009. Now with over 110,000 photographs in the catalogue spanning 34 years you might say I’m a power-user.
When Lightroom 4 was released in Spring 2012 I pre-ordered it, but as with critical applications within my workflow I wanted a few days to get my head around it before using it ‘in anger’. I’m glad I did, because quite quickly it became apparent there were major problems with it – principally, the speed of the software. It’d lag for 30 seconds switching between modules, and using the sliders to alter an image was almost impossible. I reverted back to Lightroom 3.6 which worked perfectly, as did everything else on my laptop.
Sidenote: for the benefit of the hardware nerds out there my main machine is a MacBook Pro Core i7 2.66GHz model (MacBookPro6,1), with a 512GB SSD, 8G RAM, running (at the moment) OS X 10.7.4. It’s not the most recent MacBook Pro, but it’s a 2010 model which with updates has no problems with anything else.
Time passed and I bought a new camera: a Canon EOS 5D Mark III. This wasn’t supported by Lightroom 3! Indeed, it took Adobe until version 4.1 before Lightroom could read its RAW files so I suppose you could say I was forced to upgrade. I bit the bullet, installed the software and predictably the speed problems returned: a photoshoot which I would have previously been able to sort in a lunchtime would take me 4 or 5 hours simply because of the lag when picking images to retouch (the ‘triage’ stage of sorting a shoot). The whole experience became incredibly frustrating and a solution was needed. Adobe released a beta of Lightroom 4.2 which made no difference.
Googling around led to plenty of other people with the same problems, and even Adobe acknowledging there was a problem but there was no solution. I got naffed off, and decided to do some investigating of my own. So here’s what I did, and what it fixed (touch wood, so far):
A Solution Of Sorts
The first thing I did was try and blitz the Preferences files. This has been known to work for Windows users, but of course the Mac version is different – at least that bit is. I’ll save you some time and tell you now that it didn’t do a dicky bird other than freak my copy of Lightroom out, so I ended up reinstating the plist files.
Then a friend suggested recreating the catalogue (cheers Gaz!). My LR4 catalogue came via a port from LR2 to LR3 over the past few updates. First thing I did was create a new catalogue and import everything from the old catalogue in to the new one via File -> Import From Another Catalog. It took a while but gave me a clean catalogue which sorta worked a bit better – still not massively fast, but faster.
The real paydirt came with Time Machine though. In common with a lot of Mac users I have a small external disk which I use to keep backups, handled by the OS. In addition to that there’s a sort-of journalling system which handles versions and Time Machine when the external disk isn’t connected (and incidentally will explain disk space discrepancies – you can read more about the phenomenon here).When I added ~/Library/Caches to the backup exclusions, LR4 was back to the speed I’d have expected.
To do this, click on your Time Machine icon (on the menubar, it’s the little clock with the arrow going backwards), and thenOptions on your Time Machine screen. It’ll allow you to exclude particular directories from the backup.
I can’t think of any particular downside to this as cache files are usually temporary, but I’m sure another Mac fanboi will correct me in the comments if this isn’t the case. As it is, it seems to have fixed it for me so far and I’ve been using Lightroom heavily for the past day or so. Whether it gets slower in time I don’t know and I’ll post an update if it does so. But I do hope this helps other frustrated Mac photogs out there!
I finally got sick of the blog spam under my very hacked up version of Nucleus, so I’ve shifted myself to using WordPress. Apologies if the RSS feed has just given you the past 10 articles as being ‘new’, or whatever.
Anyway, in porting all the data from Nucleus I happened upon nucleus2wordpress.pl which purported to be a hack to shift all the posts from old versions of Nucleus into, er, WordPress. It didn’t work first time since the categories system within WordPress has changed so I’ve frigged it to work with the new WordPress taxonomy system.
You can download my updated version (which works with WordPress 3.3) here – no guarantees implied etc. and it depends on your Nucleus database being the same as your WordPress database (I just copied the tables over and then removed the Nucleus ones when I was happy). Enjoy!
I’ll be back blogging soon. ‘Interesting times’ over the past six months have taken priority…
Addendum: After running this script and becoming more familiar with the concepts involved in the WordPress database, I discovered the comment counts were incorrect (even though the imported comments themselves appeared within the posts). To correct this, run the following on your WordPress database (remembering to back it up first!):
UPDATE wp_posts wpp
LEFT JOIN
(SELECT comment_post_id AS c_post_id, count(*) AS cnt FROM wp_comments
WHERE comment_approved = 1 GROUP BY comment_post_id) wpc
ON wpp.id=wpc.c_post_id
SET wpp.comment_count=wpc.cnt
WHERE wpp.post_type IN ('post', 'page')
AND (wpp.comment_count!=wpc.cnt OR (wpp.comment_count != 0 AND wpc.cnt IS NULL));
This worked fine on WordPress 3.3 and corrected the comment counters on the front page and search results.
Once again in a contract I find myself writing a small search engine, this time for an internal Intranet content management system. The client has several use-cases of which several contain words 3 characters long such as ‘cog’ and ‘net’. Additionally, they’ve got a couple of abbreviations with punctuation in (‘H&S’ being one). And once again I find myself explaining to the boss why the MySQL fulltext search doesn’t return a full set of results because (out-of-the-box) the fulltext defaults aren’t very useful.
So here’s why, and how you might be able to fix the issues (and ‘don’t use MySQL’ is not going to be an answer here, dolphinistas)…
For a start, it only indexes words of 4 characters or more, so if you’re trying to index ‘cog’ or ‘net’ then it won’t unless you explicitly configure MySQL to enable this behaviour using the ‘ft_min_word_len config parameter. Perish the thought if you’re search for your own name and you’re called Bob, because MATCH(keywords) AGAINST(“bob”) will always return nothing, as will searching for the classic album ‘Let It Be’.
Then let’s take the stopword list. Stopwords in MySQL are words which won’t be indexed – common words such as ‘the’, ‘and’, ‘or’, etc. (there’s a full list here if you’re curious). However, it also prevents indexing of words such as ‘will’ – a real bugger if you’re indexing a list of users: pity the poor guy in sales called Will who can’t find his own name. You can give MySQL an alternative stoplist in the config, or disable it completely using ‘ft_stopword_file‘.
(It’s not looking good for poor Will or Bob is it!)
Finally, we’ve got the punctuation. This one is a little bit more silly because of the MySQL boolean search which uses various forms of punctuation to specify which words are mandatory, optional, grouped, and so on: the default list is ‘+ -><()~*:””&|‘ (note that we’ve got & in there). Personally I change & to ? since I’m less likely to index with the latter, using ‘ft_boolean_syntax‘ in the config file, and things like ‘H&S’ will get indexed.
Therefore your my.cnf gets the following extra bits:
ft_boolean_syntax=+ -><()~*:""?|
ft_min_word_len=3
ft_stopword_file=
So, mostly sortable right? Wrong: these are server-wide variables and if you alter them you must reindex all fulltext-indexed tables using REPAIR TABLE (or the dreaded myisamchk if you have a lot to do). Now read that again: it’s server-wide so will affect all databases on that server… As an example, the client I’m working with right now shares the CMS database server with a MediaWiki installation which has configured in some places therefore it comes under the banner of ‘service-impacting’ straight away. It also means if you’re on a shared hosted service, no matter how much you lick your ISP it’s unlikely that they’ll change the config on a shared box full of users (with all the reindexing that implies) just for you.
I understand that the InnoDB engine is getting fulltext capability in the next major version of MySQL – about time too: I hope that will herald fulltext configuration changes on a per-database (or even more usefully a per-table) basis.
(Postscript: Nowadays I tend to tackle most of this by creating a search index from data I need to make searchable, a simple script which assembles data and removes unneeded words in a self-configured stoplist table (that way I can also write something which warns the user that not all their search terms were used). I still need to do the changes above but it affords me a lot more control over index sizes, etc.)
I’ve recently noticed a spate of problems with a keepalived install which I’m using to distribute load over several application servers, manifesting itself as:
[874439.982041] keepalived[7079]: segfault at 0 ip 0804d5fe sp bf976e80 error 6 in keepalived[8048000+20000]
[874439.987336] keepalived[7081]: segfault at 0 ip 0804d5fe sp bf976e80 error 6 in keepalived[8048000+20000]
[874439.992815] keepalived[7083]: segfault at 0 ip 0804d5fe sp bf976e80 error 6 in keepalived[8048000+20000]
[874444.956732] __ratelimit: 534 callbacks suppressed
This is using Debian 6.0.1 (squeeze) on i386 architecture.
After lots of digging around it seems the answer is simply that the module ip_vs isn’t loaded.
To fix, just do:
# modprobe ip_vs
It’ll stop the segfaults and all will be returned to sweetness and light.
(This post is intentionally Google-fodder. Hope this helps someone else with the same issues.)
A while back I acquired a dirt-cheap secondhand pair of SoundLAB Gyroscan units (G017X) for use on-stage as spotlights in combination with my ENTTEC DMX-IS computer-controlled DMX interface. The idea being that I can set them up for each song, mark a cross on the stage and that’s where the singers stand – cheap computer-controlled spotlights, yay.
There’s a couple of gotchas with this:
First, the Gyroscan DMX interface is inverted – that is to say, hot/cold are the wrong way round on the DMX pinout. While professional establishments tend to have DMX inverters to sort this stuff out I didn’t have any and it seemed to be a simple matter of popping the back off the unit and swapping the cables around. Ten minutes with a soldering iron and we’re in business.
Second, there’s no DMXIS fixture file which will deal with them properly. The Prosound Smart Scan II file comes close but wastes a few (possibly precious) channels which makes it unsuitable if you want everything on one screen. Thus I’ve put a DMXIS fixture ‘macro’ together which if you’re using Windows you can save in C:Program FilesENTTECDMXISDmxLibrarySoundlab and it’ll automatically find it:
Pan
V,0,255,
Tilt
V,0,255,
Gobo
S,0,24,Black
S,25,49,White
S,50,74,Pink
S,75,99,Cyan
S,100,124,Yellow
S,125,149,Green
S,150,174,Orange
S,175,197,WhDots
S,198,226,Rotating
S,227,255,Strobe
Rotation
V,0,255,
This seems to work fine for me and also makes the positioning trackpad work in the bottom left corner of the DMXIS control panel.
The only real comment I have on the Gyroscan units (bear in mind here I’m still an amateur at this) is that they’re a bit slow to cue up a gobo. Other than that they seem fine!
I’ve got a new MacBook Pro – one of the Intel i7 ones, 17″, nice piece of kit. I’ve also re-evaluated my external disk requirements on my desktop and bought a dual eSATA/USB RAID0+1 array for storing my Lightroom library.
Sick of running it on USB (and I’m running out of USB ports anyway) I purchased a LaCie SATA II ExpressCard 34 adaptor via Amazon. Today’s been the first chance I’ve had to switch it over as I’m working from home, and installed the drivers as per the instructions. I plugged the card in… instant kernel panic on Snow Leopard. Oops.
No problems, reboot. Except it won’t – not at all. A reboot of the machine hit a kernel panic within a few seconds, not even enough for it to bring up the OS tail. The error is: “thread wants credential but has no BSD process” – a completely unusable system.
I eventually got it down to out-of-date drivers supplied with the CD accompanying the card. The correct drivers are here, and you will not be able to boot your system unless you do the following:
I haven’t had any problems since.
(Apparently this is to do with the SIS chipset not addressing 64-bit memory space correctly, but booting into 32-bit compatibility mode doesn’t work either.)
Given my recent post on IPv4 exhaustion, I read with interest that 1.0.0.0/8 has been reallocated (to APNIC) as part of the efforts to debogon and reuse address space.
Previously there’s been discussion on which /8 should be allocated. A good read, and covers issues I’d expect to become more important as the IPv4 space is depleted. IANA keep a document on /8 RIR assignments here.
I don’t do much networks stuff nowadays, but what’s important from my programmatic point-of-view is IP address validation within user interfaces – I know at least two systems I’ve been involved in developing have IP address input validation; consequently I think you can probably expect to see older (poorly programmed) routers thinking erroneously ‘oh, that IP address is invalid!’.
According to iNetCore (via Trefor Davies’ blog) the IPv4 address space will finally be exhausted on 5th September 2011.
Available IPv4 address space dropped below 10% yesterday 19th January 2010 as announced by the Number Resource Organization. So, everyone ready to take IPv6 seriously now? 😛
(To be honest it sounds like a good excuse for an ‘end of the world’ type of party – anyone in Hollywood fancy making a disaster movie out of it?!)
I posted last week about Hotmail breaking email headers – it’s starting to be noticed a little more, and unfortunately there isn’t a solution yet. I’ve looked at rewriting the headers on my own (exim-based) mailservers but frankly it’s a pain in the arse and I don’t really want to frig something when it’s Hotmail that should be fixing it.
More links (including info on Roundcube and Thunderbird):
It’s bloody annoying, and tedious Microsoft won’t actually fix it. Do they not have a QA dept?
It’s been a month or so since release, there’s a .1 version out and the wounds of being an early adopter have healed somewhat so (gulp) I’ve taken the plunge and installed Snow Leopard on my Macbook Pro 2.33GHz Core II Duo laptop.
Oh alright, I confess I’ve not been brave enough to splat the install – instead I acquired a 500GB laptop drive and I’ve been running it on a USB adaptor. That way I haven’t blasted my 10.5 install, and can run away if need be; in short, I’m playing the safe option given all the reports of incompatibility and weirdness.
So, quick bullet points on verdict:
So not so bad yet, although I’ve only given it a day of hammer.
What else? I updated Logic Pro 9 and my copy of the B4 plugin has stopped working; also Novation’s V-station thinks it’s unlicensed if you don’t update to version 1.5.1 – this threw me for ten minutes until I googled a solution. I’ve not found a solution to the B4 issue yet. Logic 9 itself seems stable enough and the CPU load is noticeably smaller when playing complex songs (I tested with one which had 97 tracks, lots of audio and software instruments).
I’ve also dumped Parallels after nightmares with their support in the past and turned to VMware Fusion. If all goes according to plan I’ll register the copy come payday.
I have spent a substantial amount of spare time over the past few days trying to track down why any replied email from a Hotmail user shows as a blank email in my webmail (using Roundcube), and corrupted in Mulberry (regardless of platform).
The symptoms are quite simple:
This means if you are using Hotmail to send email, there’s a good chance the recipient won’t be able to read it unless they are using a Microsoft mail reader (eg. Outlook) or Hotmail itself.
You can see a typical header here (sensitive information redacted for confidentiality):
References: <BLU140-W710CB868557ABFA6D8D1FD2D20@phx.gbl>
<BLU140-W32A851FEAF0116DA3A6A60D2D20@phx.gbl>
<BLU140-W34301DB9B838BAA94D69F1D2CC0@phx.gbl>
X-OriginalArrivalTime: 13 Oct 2009 13:42:21.0449 (UTC) FILETIME=[FD2D9F90:01CA4C0A]
X-SA-Exim-Connect-IP: 65.55.116.100
X-SA-Exim-Mail-From: xxxxxxxxx@hotmail.com
X-SA-Exim-Scanned: No (on foxy.wrenthorpe.net); SAEximRunCond expanded to false
<e17ccddd5d6dc4267f8d309ae6314529@wrenthorpe.net>
MIME-Version: 1.0
--_2d718c5a-9b8c-4761-8b42-54e91ed44fd8_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Hi Joel=2C
What’s important to note here is the corruption of the ‘References’ line. I was fooled into thinking it was a SpamAssassin issue by the presence of the ‘X-SA-Exim’ lines, however these are a byproduct of the corruption itself. What has actually happened is that Hotmail has inserted an ‘X-OriginalArrivalTime’ line followed by two CRs, SpamAssassin has then thought ‘oh this is the end of the header’ and appended its own lines, effectively chopping up the header before the ‘MIME-Version’ line is reached.
The end result of all this is that standards-compliant mailers such as Roundcube, Squirrelmail and Mulberry show the message as corrupted, and this is exacerbated when anything at all touches the headers en-route. Outlook and Exchange both show the message OK, so I can see how this might have got past the Hotmail guys (who presumably have to solely use Windows nowadays).
Microsoft themselves seem to have acknowledged the problem in this support ticket from Windows Live, although goodness knows when there’ll be a fix. If it continues past a few days I’ll see if I can craft a rule to mangle the email back into a sensible format because it’s causing no end of fun issues for maillists I participate in.
Thanks to Hotmail user ess-jay, who was very patient in trying lots of different things when I was attempting to track this down.
ircnet.demon.co.uk – the first IRC server I used, which led to a major job and many friendships, acquaintances and the rest of it, is being shut down:
-BRd(~jamesr@127.0.0.1)- It is expected that ircnet.demon.co.uk will stop providing an IRCnet service very shortly (although an exact date is not confirmed). I would like to advise you that other IRCnet servers are available for users of demon’s server. These include uk.ircnet.org, irc.xs4all.nl, irc.snt.utwente.nl, us.ircnet.org, ircnet.choopa.net and ircnet.eversible.com. Thanks you and thanks to demon for providing a service for all these years.
It being a server in a larger IRC network means of course it’s not really that big news, but it’s still the end of an era.
Not that I’ve been on #dl-bar and #uknot in a long time…
Yorkshire lass, born-and-bred. Photographs stuff, writes software, makes music, cooks, dances round the kitchen. Happy lesbinum, wife, and parent to three. Proud to have a rainbow family.
Musician, composer and producer, under the monicker of The Bleeding Obvious.
Regularly seen performing live – gig list here. Been on TV and radio in various capacities.
I’m on Facebook, Instagram and Mastodon
By day I work as a systems architect for a marine technology company. More details about all that on LinkedIn.
© 2024 What Jessie Did Next…
Theme by Anders Noren — Up ↑