Restructuring Upworks marketplace

Upworks marketplace relies on freelancers accurately marketing themselves and clients (customers hiring freelancers) accurately describing their job requirements. Consequently, there is too much room for interpretation resulting mismatches and poor relevance.

Doesn’t Upwork help structure the profiles and jobs? We have a category system that clients and freelancers place themselves in. Helping to identify the type of work they’re are hiring for & working on respectively.

Part of the problem is the current categories are too broad…

This category includes so many different types of work, such as iOS, Android, Cross-platform (ionic, xamarin), etc.

I tried to solve this by improving keyword suggestions to help them refine their lists further. Which worked great… however, it’s limited to search and doesn’t resolve the systemic issues.

What are the systemic issues?

Starting with the client ones, we surveyed all the clients that didn’t hire but posted a job in 2016. The results of the survey highlighted two main reasons clients choose not hire.

Supply - They didn’t get enough applicants.
Quality - They didn’t get qualified freelancers.

These reasons didn’t line up with our quantitative data; we’re seeing thousands of available freelancers. Freelancers with experience working on jobs similar to the ones those clients needed. I did some more digging and found that although the search was returning relevant freelancers. It was difficult to see why they are relevant, at a glance on their profile.

This issue was prevalent, profiles were all over the place. It’s difficult to figure out what a freelancer specializes in or their experience with that specialty. The quality issue seems like it might be, at least to some extent, a perceived relevance issue.

We implemented a design on the search page that displayed the number of relevant jobs a freelancer worked on. Then on the profile, I added a filter to show only their relevant work history. While this solved the perceived relevance issue, it was another band-aid.

What issues are freelancers facing?

Freelancer issues fall into two buckets. How clients inaccurately describe their jobs, and freelancers having to cram all the work they can do into one profile.

There’s little to no guidance on the job post form. Clients have to know a decent amount about the type of work they’re posting. It’s easy to mess something up, miscategorize or include skills that aren’t relevant. It can be frustrating for a freelancer to spend time applying for a job that looks relevant, only to find out the client needs a different skill set.

Unfortunately because of how many jobs a freelancer has to apply for, to get one response. It’s very common for freelancers to quickly skim jobs until they get a reply, exacerbating the issue. That feeds into why clients say they aren’t seeing relevant freelancers.

When it comes to the profile, Upwork has a rule limiting freelancers to one profile, to prevent spamming. Resulting in freelancers including all the types of work they can do in one profile. They do this to appeal to as many clients as possible

Upworks algorithms rank freelancers by relevance, determined by title, overview, metrics and anything else on the profile. If a freelancer wants to appear in search, they have to include it, same as SEO. You can see the result above, a massive amount of freelancers look like generalists, they know every language and can do every job.

Generalist profiles are, at least, part of why clients have a perceived relevance issue; we don’t give freelancers an alternative. They are incentivized to look like generalists, to garner as much attention from as many clients as possible.

Lastly, Upwork needs insight into the supply.

Due to a lack of insight into what a client is trying to hire for and what is the primary type of work a freelancer can do. Upwork doesn’t have a good understanding of supply & demand in the marketplace.

Knowing this could make it a lot easier for marketing to target acquisition and help product focus on improving specific features for those customers.

A Quick recap of the systemic issues

Clients - Need to see the types of work freelancers can do, separately not just mixed altogether. So they can better identify the relevance to their job.

Freelancers - Need more clarity on specifics of a job. Need a way to separate types of work they do on their profile.

Upwork - Needs way more visibility into supply and demand within the marketplace.

How can we design a system to solve these issues?

The first thing was to work with the team to get aligned with a newly unified taxonomy, one that worked for all sides of the marketplace. We knew the framework that on search was working well for clients, so I started there.

Say you’re looking for an iOS Game Developer. When you post a job for that role, this is the existing category for that type of work.

If you’d try to bucket that job, you might say that iOS Game Development is a category. You’d be right, but it’s not the only type of category at that level. Game development has a different skill set then general Application development. They both fall under the umbrella of iOS Development. We call this a service, we defined it as a type of work or a job role.

As I mentioned above, we can bucket the different types of development skill sets under iOS development. We call these Specialties; it’s something that requires even more specific domain knowledge.

How were all the services & specialties chosen? For now, we used search queries and frequency to create the first version. Once launched, it will be tweaked based on what is being requested by clients and freelancers.

We knew this framework was working for clients because it was a successful feature launched on search. What about freelancers? We took the opportunity to see if freelancers thought about their services and specialties in the same way.

Freelancer co-design interviews

To figure out if services & specialties made sense to freelancers, we set up some interviews. Asking them “if they could, how might they organize the types of work they do into buckets” This way we wouldn’t bias their thinking, with our own.

An example of how freelancers broke up their profile by type of work.

Through these interviews, we were able to confirm that freelancers and clients both think alike, regarding services and specialties. Now we have a direction on the taxonomy the next step is to figure out how to implement it.

Utilizing the new taxonomy to solve the systemic issues

Okay if you recall, the shared problem for clients and freelancers were profiles containing a mix of all their work. Making it unclear what their specialties are and making them look like generalists.

Why not filter the profile by specialization? It would create a lot of profiles. Whereas we wanted to limit the number of service profiles a freelancer could add to 5. In order to get a freelancer to focus only on their primary specialties.

How do freelancers create these profiles? Ostensibly through a form, It’ll be part of a separate case study.

Now that we have these new fancy service profiles, I needed to figure out how to get the client to the right profile when they search.

We based the new taxonomy on a query suggestion system, and I created last year. In the short-term, I also used the same UI, only updating the labels. The long-term goal would be integrating the new taxonomy and old categories so a client could navigate up and down the taxonomy quickly, similar to Amazon.

Last part needed for the system to work is updating the job post. Below is the current job post, the only field that needs to be updated is the category selection.

Due to the number of services, it would have been frustrating to reuse the category drop down. We knew the first design wasn’t exactly ideal, and I wanted to use an approach that showed the top three matching services based on the title of their job. The issue is we needed data before we could train an algorithm to do that. Eventually, we’ll update the design once the algorithm is ready.

Now we have the system designed, what was the feedback from customers?

We did a bunch of concept testing early on we knew the concept appealed to freelancers and clients. Now we have a design we started usability testing to see if the concept integrated well into the product.

“it’s very helpful to have people that have very specific experience types.“

“I see you’ve color-coded and changed the type of job. That type of job, I don’t really care about too much, it’s like how many hours has he worked, was he good, is there any feedback.”

“Specializations, yeah I wouldn’t know what that meant.”

“It was a little confusing to me, only ‘cause I’m new to the game (hiring on Upwork)”

“give option of old format, vs. new format - I can see how this would help someone who has no clue, but it was ridiculous when I have a quick job to post“

Job post picker was a big issue for clients. We designed the solution to improve this experience, and it is a matter of training the algorithm.

What are the results? What’s Next?

Right now the jury is still out metric wise; Directionally relevance matching is trending upwards. We’ve heard mostly positive feedback from clients and freelancers so far as well.

Next step is to roll out more services, right now its only for mobile development. After that I’ll be working on customizing the profiles, search filters, and job post for each service.