Remembering all of my LinkedIn connections with Polarity

polarity-linkedin-2-01

Like many of us in the cyber industry, we leverage LinkedIn to maintain our contacts and track people we’ve worked with. One of the most useful things I’ve done with my LinkedIn contacts was to import them into
Polarity. Now, when I read the news, look at conference speakers, participate in industry Slack channels, or even scan a list of investors or board members for a company I am researching, Polarity highlights who I “know” from LinkedIn. This makes my time more productive and my connections and time spent previously with contacts, much more meaningful. In this blog post, I will show some examples of what this integration looks like and how to export, clean and import the data into Polarity.
Polarity

The Polarity solution is sold to enterprises on a per-user annual license. A server is configured with static and dynamic data and users with Polarity clients can interact and add new data. Team members can on-the-fly select text (such as a person’s name or an IP address) and add an annotation to it. Unlike taking a note in your orchestration, chat, ticketing, or other knowledge management system such as OneNote, the Polarity annotations are instantly integrated with all your tools and the whole team. You or your team members continue to do their job and as they see relevant data on their screens, it will be highlighted automatically.

Although Polarity is an overlay and works on top of all of your existing tools and work flows without any integration, I still connect Polarity to some existing or dynamic datasets with open source “integrations” built on top of the platform. Polarity integrations include cyber security technologies such as Virus Total, Recorded Future, Phantom and ThreatConnect as well as CRM and IT tools like Salesforce, Confluence and DiscoverOrg.

In my lab, I run Polarity configured with a variety of integrations including AlienVault and Virus Total. Here is a screen shot of my Carbon Black instance.

carbon-black

Both IP addresses colored purple indicate a Virus Total hit. Hovering the mouse over one of the highlights opens up the details of what was known about Virus Total.

Below is a screen shot of a command line
ssh session where I connected to one of my test Ubuntu virtual machines.

RonLab

Each IP address was highlighted orange because I had uploaded an asset list of names and descriptions of my IP addresses used in my lab and home office.

LinkedIn and Polarity

My ideal use case would be for the Polarity server to dynamically compare names to my LinkedIn contact list, but LinkedIn does not support this type of API driven lookups. They don’t even allow you to automatically sync data to places like
FullContact – an app that Polarity does support with API lookups. Instead, this example focuses on the export of LinkedIn data, massaging that data and importing it to Polarity.

My target use case is pretty simple – I want to manually export my connections from LinkedIn, import them into Polarity and then when I see a first and last name of someone I am connected with anywhere on my screen, have Polarity highlight it so I can quickly remember who they are and where they currently work at.

To illustrate LinkedIn data usage, I opened up the table of contents for the list of authors who contributed to the
Tribe of Hackers book. I had configured Polarity to highlight my LinkedIn connections in green. Below is a screen shot.

TribeOfHackers

This is really cool because I quickly can see that I have some sort of connection to people in this book.

Polarity leverages computer vision to perform recognition of desired patterns. One of the issues with hooking in data from LinkedIn is my method uses an exact match. I’ve been friends and linked in with
Jeff Man for several decades, but the Tribe of Hacker’s book lists him as “Jeffrey Man” and his LinkedIn profile is “Jeff Man”. Polarity allows you to have multiple tags and entities, and on-the-fly data entry for easy updates, but my particular use case is to re-import my entire LinkedIn contact list periodically since I want to capture where people are working at right now as well.

Here is another example from an upcoming BSides conference.

BsidesNova

Exporting, Cleaning, Importing and Working with LinkedIn Data

Getting LinkedIn data into Polarity is really simple – export the data, clean it and then import it.

LinkedIn has a
process to export your list of connections. You get notified via email when your data is ready. This is downloaded in a ZIP file which has a single CSV in it with your columns selected called Connections.csv.

If you look at the data in a text editor, you will see lots of issues with the data, such as inclusion of UTF character sets, double and triple quoting of names and title (i.e, my name is “’Ron Gula’” not Ron Gula), names including a title and initials such as Dr. R. Gula and so on.

Many of these can be fixed within a spreadsheet editor, but I wanted a simple command that could clean up all of my issues. I settled on two Linux command line tools –
iconv and gawk.

The following command uses
iconv to convert the Connections.csv file to ASCII and delete any characters it can’t print.

iconv -f utf-8 -t ascii -c Connections.csv > Connections-Ascii.csv

The gawk command is more of a beast, but I felt it was less complex than doing a python script.

gawk -vFPAT='([^,]*)|("[^"]+")' -vOFS=, '{gsub(/,/," ",$1); gsub(/,/," ",$2); gsub(/,/," ",$4); gsub(/,/," ",$5); gsub(/"/, "", $2); gsub (/"/, "", $4); gsub (/"/, "", $5); split($2, a, " ", seps); split($1, b, " ", seps); c = b[1]; if (length(b[2]) > length(b[1])) {c=b[2];}; if (length($4)>5) { print c " " a[1],$4,$5}; }' Connections-Ascii.csv | tr -d '"' > ConnectionsClean.csv

This command reads the Connections-Ascii.csv we created with the iconv tool and outputs it to ConnectionsClean.csv. The command line performs a variety of cleanups including removing names of users that put a comma in their names such as “Ron Gula, CISSP”, removing quotations around names and jobs and dropping names that are shorter like “R Gula”.

The file
ConnectionsClean.csv should be moved to a location where it can be uploaded into the Polarity user interface. In my case, I moved it to my main OS X system and imported all of my contacts. Polarity gives you a preview of the data you are loading and also a row count so you can verify that 100% of your data got included.

Conclusion

My ideal use case is to have this sort of work done in an automated fashion. Unfortunately, until LinkedIn makes it easier to automate the export of your connection list, this will be a manual process or require the introduction of third-party tools which have full access to your LinkedIn data.

But the results I’ve had are really worth the integration effort! Now as I do my work and process email, read news, watch security talks, look at companies, .etc. Polarity enables be to be easily reminded of how connected we all really are and how small the world really is.