I wanted to remove Evernote from Spotlight.

It is easy enough to remove a Folder from Spotlight. Just go to System Preferences » Spotlight » Privacy, and drag any folder from Finder into the list. That folder is no longer indexed.

But the folders for Evernote, reported by others to meet this need, are ineffective. (I rebuilt the Spotlight index and they came back!)

• ~/Library/Application Support/com.evernote.Evernote
• ~/Library/Caches/com.evernote.Evernote

So I went looking for the importer for Evernote:

mdimport -L

That output includes:

/Applications/Evernote.app/Contents/Library/Spotlight/EvernoteSpotlightImporter.mdimporter

So I deleted that Folder and then rebuilt my Spotlight index with:

sudo mdutil -E /

Evernote does not encrypt data at rest. Evernote does not apply “zero knowledge” encryption (where data in encrypted on the client and the provider has no ability to decrypt the customer’s data).

I’m sure Evernote is a fine company, but I see no reason to believe that their security team is any sharper than their development team, and there have been many defects in the application through the years. Eventually, I can expect a breach. I used to use Evernote as my file cabinet. I’ve decided it is simply irresponsible of me to continue to store tax records, medical records, bank statements, etc. in a cloud-based product which does not apply zero knowledge encryption.

Evernote, I’m leaving you. It will be a difficult separation, but I’ll manage somehow.

Some things I really liked about Evernote:

• I use a Mac. My wife uses Windows. We both need access to our file cabinet. Evernote made that easy, and kept us synched.
• The file cabinet metaphor has its limits. Not all data fits into a single spot in a hierarchy, plus lots of my stuff gets stashed in ‘the big pile.’ Evernote’s search tools just work better than Spotlight for me.
• Separation of documents from app data and programs. Spotlight likes to show me every file. 99% of the time, I’m searching for a document. (Document includes saved web pages.)

I’m going to capture my strategy for leaving Evernote here. I’m going to capture the problems I encounter. I’m going to capture the reasons I do what I do, so that I won’t forget and revert.

##Requirements:

• The main parties:
  • Bad guys must not be able to access documents. (Secure)
  • My wife and I have to be able to find the right documents quickly. (Accessible)
  • I/wife/executor should be able to get to it if my house burns to the ground and I get nothing out. (Persistent)
• Must be able to export the documents (unless they live in the file system as well-titled files), in case the app/service is withdrawn.
• Access when I’m away from home is a nice-to-have, but not a requirement.

##Alternatives:

• Continue with Evernote as is. Why rejected: Unacceptable security risk of unencrypted data on server. Meets all other requirements.
• Continue with Evernote, making all notebooks local-only. Why rejected: Not accessible to wife on daily basis. (Executor could get it from my off-site backups.) Meets all other requirements.
  • Note that simply copying the database from my Mac to my wife’s PC. Windows and Mac Evernote use different database formats.
• Store documents in the file system. Plain old notes (as would be composed in Evernote) get stored in RTF or HTML or Markdown for portability. Not as accessible. Meets all other requirements.
  • My local file system is encrypted, as is my wife’s - providing security.
  • My local file system is backed up off-site with a service - providing persistence.
  • Spotlight (Mac) and Windows Search are just not as good at finding the right file as Evernote’s search and display.
• Store documents in the file system, auto-import new/updated documents into local-only Evernote. Synch files with my wife. Never update an Evernote document from the Evernote UI.
  • If I could pull this off, it might meet all requirements, with the down-side of doubling my data size. I currently have 5 GB in Evernote. That’s not a terrible cost.
  • Or turn this around and only update the documents in (local) Evernote, export the recently updated items, and sync via the file system to my wife’s PC.
• Migrate Evernote to OneNote. OneNote meets the security requirement technically, but not practically; it misses the export requirement.
  • Does not store data locally on the Mac. That’s a big miss on accessibility. I’m not going to rely on Internet availability to get my documents. If MS drops the service, I lose Mac access.
  • You can password protect a OneNote “notebook section.” Microsoft says they can’t decrypt it. You may have to protect it using the desktop client - it isn’t clear that www.onenote.com can add the password.
  • You have to remember to password each new section and you have to enter the section’s password to open it.
  • “Password-protected sections aren’t included in notebook searches. To include a protected section in a search, you’ll have to unlock the section first.”)
  • Export can create only a single file for exports. i.e. You can export a single note to a document or you can export an entire notebook to one document.

What I really want is the Evernote user interface on top of the native file system.

##Things I’m Trying In Order to Make Spotlight a Better Fit

If I’m going to use Spotlight, I need to make it find the documents I’m looking for most often.

• Change indexing:
  • Exclude folders:
    • I’m going to exclude most folders, and use Spotlight solely for folders where I store documents. For other files, I’ll just use ‘find’ in a Terminal window.
    • I’ve read that Spotlight is hard-wired not to index source code.
  • While I’m transitioning, I want to remove Evernote from Spotlight but keep the data as a backup. These were ineffective at removing Evernote. See this story.
    • Exclude folder ~/Library/Application Support/com.evernote.Evernote
    • Exclude folder ~/Library/Caches/com.evernote.Evernote
  • Ultimately, I deleted the .mdimporter file from /Applications/Evernote.app/Library/Spotlight
  • Note: If you want to rebuild the Spotlight index: sudo mdutil -E /
  • Ugh. Apparently, it won’t index .html files unless you have XCode. Install XCode.
  • Here’s how to add a file type to be indexed, using the rich text importer: http://robin.wenglewski.de/posts/2015/05/26/osx-markdown-indexing-knowledge-management/
• Change searching:
  • Go to System Preferences » Spotlight » Search Results
    • There’s not really enough granularity/clarity here. Make some guesses. Exclude everything except documents. I’m searching only:
      • Applications
      • Developer
      • Documents
      • Folders
      • Images
      • Movies
      • Music
      • Other
      • PDF Documents
      • Presentations
      • Spreadsheets
    • You could control the order in which items were displayed in the search results by re-ordering this list in older versions of OS X, but cannot do so any longer.
  • If you want better control over the display of search results, start your search with Alt-Cmd-Space instead of Cmd-Space
  • There are tales that you can edit com.apple.Spotlight.plist in ~/Library/Preferences (with an editor that understands binary plist files such as BBEdit or TextWrangler) and rearrange the dict items in the array. You may have to reboot immediately afterward. You may have to do this in single-user mode/special-boot-mode.
  • Alfred doesn’t find things I think it ought to find. (Yes, I used the “in” keyword.) It just doesn’t find things that Spotlight does.
  • Spotlight puts my .html documents under the category “Developer”. I want them listed as “Documents”. I can’t find a way to make it do that. I looked for about a half day.
  • This will find all your mdimporters (and it is a manageably sized list): find / -name \*.mdimporter -print 2>/dev/null

Synchronizing

I used to use Evernote to sync my documents with my wife. Now that my documents are in the file system, I need another method. Mostly, I am the family member who files things, so I can live with one-way replication. That’s good because Unison 2.48.4 (on my wife’s PC and on my Mac) has a bug where it always creates files with 600 permissions. (Yes, I set the options and umask appropriately.) If they fix it, Unison would allow me to use 2-ways synch, but for now I’m using rsync.

• I installed Cygwin, rsync, ssh, and unison I mostly used this as a guide for setting up ssh.
• I used this command line on my Mac to sync the files:
  • rsync -avz –delete –exclude ‘.DS_Store’ /Users/kevin/Sync/Docs kevin@suedell:/cygdrive/c/users/susan/sync > “/tmp/sync-to-sue.log.$mm.$dd” 2>&1 &

• I will eventually set up a cron task to run this script daily. (For now, I simply launch it from .bash_profile.)

        #!/bin/bash
        mm=`date '+%m'`
        dd=`date '+%d'`
        yyyy=`date '+%Y'`
        marker="/Users/kevin/.sync-to-sue/done-$yyyy.$mm.$dd"
  
        try_to_sync() {
            if ! ping -c 1 -t 1 192.168.8.20  ; then
                echo "Unable to ping Sue"
                return 0
            fi
            echo "Syncing to Sue"
            rsync -avz --delete --exclude '.DS_Store' /Users/kevin/Sync/Docs kevin@suedell:/cygdrive/c/users/susan/sync > "/tmp/sync-to-sue.log.$mm.$dd" 2>&1 &
            if [ $? -eq 0 ]; then
                # nothing
                echo "Launched rsync to Sue"
                touch "$marker"
            else
                echo "Failed to launch rsync to Sue"
            fi
        }
          
        mkdir ~/.sync-to-sue 2>/dev/null
  
        if [ -r "$marker" ] ; then
            echo "NOT syncing to Sue. Already happened."
        else
            echo "Starting sync.  Did not find $marker"
            try_to_sync
        fi
  

Investigating:

• I need to find un-OCR image PDF files and run them through OCR.
• I’d like to put all my documents in ~/Documents/really-documents and all my data in ~/Documents/really-data. One of the nice things about using Evernote is that it contained solely documents. However, I also want to put my tax forms (as PDFs) in the same directory as my TurboTax data files. I can’t have it both ways. (I’m not going to use symlinks, links, or Aliases – they are too fragile.) Which is more useful?
  • Another example: Do I want my readme.txt with its source code or in ~/Documents/documents-really? I think this one is obviously “with the code,” but how do I write a simple rule?
  • The “file cabinet” metaphor almost works. If I would put the printed doc in a file cabinet, it should go in really-documents. Source code would never go in a file cabinet (except, perhaps, with a patent filing or a copyright filing). Data lives on the computer in really-data. But what about things like “Notes on how I am currently doing GTD”? That’s clearly a document, not data, but it isn’t a static document, so if I filed it, I’d be planning to pull it back out and revise it.
  • It seems like maybe:
    • Static documents from others go in really-documents.
    • Documents that have multiple viewers go in really-documents.
    • Source code does not go in really-documents. It isn’t data. It goes somewhere else.
    • Data (typically not 1-to-1 mappable to a document) openable only by one app goes in really-data.
  • What are “Documents”?
    • .pdf
    • .doc and .docx and .rtf and .rtfd
    • .xls and .xlsx and some .csv
    • Visio diagrams
    • Most .txt files (text documents)
    • .html and .mht files (and the standard variations on those names) that are not part of a web site I’m developing/analyzing
    • Emails extracted from an email system for a permanent record. (May include .html or Outlook .msg files.)
    • .opml that are really outlines/checklists
  • Non-documents:
    • source code
    • .html, and .mht files (and the standard variations on those names) that are part of a web site I’m developing/analyzing (they are really just a special type of source files)
    • TurboTax and Quicken data
    • Some .csv
    • Occasional .txt files (text data)
    • Zip files, tar, gzip, bzip, etc.
    • Emails in an email system.
    • Backups (data and document backups)
    • .opml that are really my podcast/RSS (or similar) state
  • Maybe this makes sense:
    • grep-able stuff goes in really-documents, unless it is source code
    • Non-grep-able stuff goes in really-data
    • Source code goes in source-code
    • What else exists?
      • Audio recordings, made during project meetings.
      • Are eBooks documents or data? I think maybe they are documents, but they are really big and may be un-searchable. Are my Kindle/Calibre books data or documents?
        • In favor of declaring them data:
          • I have about 3GB (2017). If I make them documents, that will consume much of my cloud storage.
        • In favor of declaring them documents:
          • If someone writes an mdimporter, I can search them.
          • They are, logically, documents. What is a physical book if not a large document?
          • Most of them can be rendered by more than one reader app. Render-by-multiple-apps is a ‘document’ heuristic.
          • I might want to share them with my wife. She’ll think ‘document’.
        • Decision: I’m calling them documents, but I’m putting them in Sync/Books, parallel to Sync/Docs, because I may want to:
          • Synch/not-synch them, separate from personal documents.
          • Search/not-search them, separate from personal documents.
          • Note: We put personal paper documents into a file cabinet, and books in a bookcase. They are stored similarly, but in different containers.
      • I’m going to treat source code like eBooks. It goes in Sync/code, as a peer of Sync/Docs.
      • What about a spreadsheet showing health data like blood pressure and blood chemistry?
      • What about a camping checklist?
      • Where do I put a Drupal web site in development? Clearly not a document. Not data (except for the MySQL data). Not really source code.
      • Music
      • Movies + TV + Video
      • Ableton Live files
      • Punch Pro home plans
      • Genealogy data
      • Audio books
      • Tiddlywikis
      • Virtual machines
    • What about things like easy-rsa, where I have a mix of code and data? I think there are 3 kinds of programs:
      • Apps (foo.app)
      • bin (stand-alone executable files and executable scripts)
      • Packages: Things like nirv and easy-rsa. Do I want to park these under ~/Packages or ~/bin? What about packages that I “make install” after un-tarring them into a directory?

Home Folder:

• Applications
• backups
• bin
• Desktop
• Documents (don’t put anything here unless you must)
• Downloads
• jumbo
• Library
• Movies
• Music
• Packages (or should these go into bin?)
• Pictures
• Public
• Sync (This is where I really store documents.)
• temp

I know how to add users to old Windows. Microsoft changed it and made it more complicated with Windows 10. Here’s how to find the old add-user dialog.

• Start » Run » control userpassword2
• Press Add
• Click “Sign in without a Microsoft account (not recommended)” down at the bottom of the dialog.

Candelabra means e-12 (or E12) socket/base.

• Tasks are tangible (physical, visible)
• Lists
  • Inbox
  • Next Actions (NA)
  • Waiting for
  • Projects
  • Later
  • Someday
  • Focus: Add a star to it if I really need to do it before I knock off for the day (Sat+Sun = 1 “day”).
    • Tag it with “weekend” if I can’t star it for today, but I want to consider starring it on Friday night.
• Calendar

Inbox:

• Barrier to entry is low
• Process in sequence
• Actionable? Toss it, file as reference, someday/maybe it, schedule to revisit (once) OR…
  • Can i handle it in 60 seconds? Just Do It.
  • Capture the tangible NA; if it is a stand-alone, capture it in NA List; else capture it in a project
  • NA list should be exclusively for “do ASAP, if in context”
• Blocked?
  • Move to Waiting For and put a nag/fup date on it
• Is it a Project (multi-step)?
  • Create a project list, and capture the project’s NA
• Contexts
  • Important to David Allen. Not so much to me.
  • I care about work role vs personal (“kpk”) role
  • Tag actions/projects with role (“Area” in Nirvana)
• Someday List is a problem for me because it is huge.
  • Put some stuff into Later. Higher priority? Really committed?
  • Put the inconsequential or the lower priority or the really just “maybe” stuff into Someday.
• Don’t dilute the calendar. Only put hard commitments there. Will happen at that time.

Weekly Review - Check:

• Every project has a NA
• For every NA, will I really do it this week, time permitting? If not, Someday it.
• Are my NAs things I can do in one sitting? Subdivide, if not.
• Later list:
  • Move things to NA list, if NA is true.
  • Move to “xxxx” list, if I’m not realistically committed to do it in 6 months but I don’t want to give it up.
  • Move to “give it up” list if I’m giving it up. Capture why.
• Someday list:
  • ONLY process this list once each quarter (March, June, September, December)
• What can I do with my boys for family time this week?
• Review my Read reference list
• Review my Covey Roles reference list
• Clean-up/migrate any other Reference lists. Do they really need to be in Nirvana?
• Are there any big picture things I need to meditate on how to improve/handle?
• “recognize and excel in what really counts — and to aim for less than perfect in everything else. If you’re shocked and feel like this seems completely unfair, I’m guessing that you probably performed very well in school where perfectionism is encouraged.”

Read Reference list:

• Entertainment
• Enrichment/self-improvement
• Work (ATT or kpk work). Move these to project or stand-alone if I’m going to get to them this week

Misc Notes:

I never can remember these:

• Move and overwrite = just drag and drop.
• Move and insert = shift drag and drop.
• Move and copy = control drag and drop

I hate “ln -s”. I can never remember which to put first – the name I’m creating or the name which already exists.

ln -s {target-filename} {symbolic-filename}

or

ln -s {thing-that-exists} {new-thing-that-will-exist}

or

ln -s {old-thing} {new-thing}

You’re thinking, “I’d like to use VPN, but I can’t afford a Cisco AnyConnect license.” I’ll walk you through setting up free OpenVPN software on a $50 Raspberry Pi computer.

Benefits:

• Keep your data private when using public WiFi.
• Access files and servers on your home LAN when you’re away.
• Practice principles which apply to the VPN at work. (Qualify as an ‘Experienced VPN Technician!’)
• Impress your friends with your technical wizardry. ;-)

Note: I’m installing OpenVPN on a stand-alone computer; some consumer routers will run OpenVPN. If you choose this route, you must still generate the keys and certificates, but your router will include GUI features to ease configuration.

Return to Surf Safe at Starbucks

Here’s where I collect my ‘learnings’ on setting up and operating OpenVPN.

Handy Command Lines

Restart OpenVPN server, to re-load config file:

sudo service openvpn restart

OpenVPN Always Connects Twice or Connects, Drops, Connects, Drops, Connects…

I set up my OpenVPN system and went to make my initial connection. It connected! And dropped, and connected and stayed connected. In fact, every time I told the client to connect, it would connect, then drop, then reconnect. Weird. Usable. A little slow. A mystery.

I’m using a Mac client, but I’m confident that the problem would also occur with a Windows or a Linux client, given what I found.

Here’s what I saw in the server log (loglevel=4):

Sat Feb 11 15:49:01 2017 us=98029 user-key01/192.168.1.11:49282 PUSH: Received control message: 'PUSH_REQUEST'
Sat Feb 11 15:49:01 2017 us=98346 user-key01/192.168.1.11:49282 send_push_reply(): safe_cap=940
Sat Feb 11 15:49:01 2017 us=98746 user-key01/192.168.1.11:49282 SENT CONTROL [user-key01]: 'PUSH_REPLY,route 192.168.1.0 255.255.255.0,redirect-gateway def1,route-gateway 10.16.0.1,ifconfig 10.16.0.3 255.255.255.0' (status=1)
Sat Feb 11 15:49:02 2017 us=741673 user-key01/192.168.1.11:49282 Connection reset, restarting [0]
Sat Feb 11 15:49:02 2017 us=741962 user-key01/192.168.1.11:49282 SIGUSR1[soft,connection-reset] received, client-instance restarting

The first observation here is that the connection dropped immediately following the PUSH_REPLY. Something in that push is breaking the connection. The first step in debugging is to start commenting out the ‘push’ lines in your server config, one at a time (with a ‘sudo service openvpn restart’ after each change) and re-testing each change. I discovered that commenting out the following line caused it to stop this behavior:

push "redirect-gateway def1"

OK. That config tells the client to route all traffic via the VPN. I want to do that. I don’t want my laptop talking to other laptops on the coffeeshop’s open WiFi. But it breaks my VPN. Hmmm…

The kicker here is that my test lab has my laptop on IP address 192.168.1.11 and my OpenVPN server is on 192.168.1.10. In other words, my test VPN server is on the LAN with my laptop. I’m using a TCP tunnel, so I have a TCP connection across my LAN from my client to my server. When I “redirect-gateway def1” (whether I push that from the server or I put it in my client config), I break that TCP connection.

The bottom line is that “redirect-gateway def1” is incompatible with having your VPN client and server nominally on the same LAN.

OK. So how do you test when your client and server are on the same LAN? Go to your external firewall, and open a pinhole to your test server, and connect to your server via its EXTERNAL address.

For example, if your external IP address is 12.34.56.78, and your test server is on 192.168.1.10 at port 443, configure your external firewall to pass traffic from 12.34.56.78:443 to 192.168.1.10:443, and change your client config to use “remote 12.34.56.78 443” instead of “remote 192.168.1.10 443”.

Interestingly, “persist-tun” on the client turns out to be relevant. If persist-tun is present, the reconnect will succeed, giving you the symptom, “OpenVPN connects twice for every attempted connect via the client UI”. If persist-tun is absent, your symptom will be, “OpenVPN in connect/disconnect loop” or “OpenVPN Connect, Disconnect, Connect, Disconnect…”

Routing problems

• If you can get to 10.16.0.1 (or 10.8.0.1), but you can’t reach the Internet, check to see that /etc/openvpn/firewall-rules.sh got executed.

Address Problems

• If your VPN usually works, but one day your traffic won’t flow, check your IP address before connecting to the VPN. If you followed this tutorial, your VPN IP address will be 10.8.x.x or 10.16.x.x. If you connect to a WiFi which happens to use those subnets for its address, the results will not be pretty. Personally, this is part of why I set up two VPNs (one TCP and one UDP). If they’ve got different address ranges, you can just switch to your other VPN.

Incidentally, the last time I connected to ATTWIFI, it assigned 10.20.x.x, but I don’t know whether that’s consistent. Many small-time WiFi will assign an address from 192.168.x.x.

Return to Surf Safe at Starbucks