It's that time of the year again when my current company kicks off our annual hackathon event. We refer to it as "Guild Week" in our department, and this marks my second participation.

I've been with Synacy for nearly two years now. During my first participation, our team's project emerged as the winner for the People's Choice Awards. Therefore, as I join the event for the second time, I am positive and optimistic for a successful outcome.

The team rosters and designations were announced by the managers two weeks in advance, providing ample time for the teams to prepare for the weeklong hackathon. I am part of a team comprising three full-stack developers (including myself), one network engineer, and one IT apps developer. While one of our members will be joining remotely, the rest of us will be participating in the face-to-face setup at the KMC co-working space in Cebu. I had to travel from Southern Leyte to Cebu for the event. The company arranged hotel accommodations for those of us who traveled to Cebu, conveniently located near Cebu IT Park, the event venue.

A week before the hackathon, I initiated conversations to formulate ideas for our project. Each team member was encouraged to share challenges related to work processes or suggest manual tasks that could be enhanced through automation. Queennie, our network engineer, and Harold, our IT apps developer, quickly contributed their ideas. Feeling the need for someone to organize these concepts, I stepped up and proceeded to arrange their submitted ideas on a Notion page, providing a common ground for all our activities.

Day 1 - Pitching day

We settled on our team name: "TEAM BANGAN" (Bundle of Amazing Neo-intellectual Gangstas w/ Appealing Niche). The name was intended to be humorous, meaning "weighting scale" in the Cebuano dialect. I suggested it just a week before the event, and we decided to go with it.

February 19 was scheduled for pitching ideas and for teams to decide which ones they would be developing. Our team was still undecided on the problem we would be solving. So, we had an emergency meeting at 12:30 pm to finalize our decision on what to build and to prepare our presentation for the pitching demo. We ranked each idea using the action priority matrix to identify those that would require the least effort but have a high impact, ideally selecting ideas meeting this criteria for our hackathon project.

After ranking the ideas, we chose to build the project titled "Automated Zendesk Ticket Reactivation for Scheduled Tasks (AZTiReSchT)." The NOC team from Rise faced an issue where they needed to reactivate tickets scheduled for exact work times. Although they already had a Zendesk automation for this, the problem was that they were restricted to setting the automation to reactivate the ticket only a day ahead. This was suboptimal for them, as they wanted the tickets reactivated 15-30 minutes ahead of the exact scheduled time.

We adjusted our slides and gained everyone's agreement on what we were going to build. During our pitch in the afternoon, we presented all our gathered ideas and concluded by announcing that we would be building "AZTiReSchT" as our hackathon project.

After all the teams pitched, our team regrouped to discuss the approach we should take to tackle our problem. At this point, I had zero context on what Zendesk was and its use cases. We consulted with senior developers and managers to determine the path we should take regarding testing and deploying the application.

Day 2 - Research

On day 2, the focus was primarily on researching Zendesk and delving deeper into its functionalities. It's a completely new software for me, and I'm still trying to wrap my head around it. I thought it would be beneficial to start using it to get more comfortable. Quennee provided access to a Zendesk account connected to the production line where the network department operates, leading to our first dilemma:

How do we create an environment for testing the hackathon project?

There are custom-made fields on the production line, and setting up another environment for the hackathon would require significant effort. We decided to use the Zendesk production line for the hackathon but with caution.

We set up a group of multiple test tickets and tagged them with a special tag ("guild_week"). The plan is to use only these tickets for performing CRUD operations for our hackathon project. Still, the risk of making a mistake on production is 50/50.

But in implementing your application logic, wouldn't that mean having to test the logic multiple times?

Yes, in order to see if our code works, we need to actually test it. On the second day, I also researched an SDK for the project and found the "Zendesk Java Client," a Maven project enabling CRUD operations to Zendesk as if it were regular Java.

Since we practice unit testing in our day-to-day work, I proposed leveraging unit testing to address this issue. We decided to make API calls to Zendesk only during our final testing. With my teammate, Junsaku, we collaborated with other developers to make unit testing on our application work. This way, we wouldn't have to make API requests to the production Zendesk every time we test our implementation, leading to another crucial question.

How would we deploy the application?

We researched and brainstormed different ways to implement and deploy the application, with two ideas standing out:

• One involves building the project's logic on top of an existing microservice application and using a cron job to trigger the flow at the desired time. This is straightforward as we won't be building it from scratch, just adding new code on top of a stable microservice.
• Another idea is to deploy our logic inside a Lambda on AWS and have another service trigger an event at the desired time. Since we estimated the code to be minimal, we run it as a service whose instance is only spun up when needed and shut down when not. This makes more sense as we are not incorporating Zendesk functionality into another service that encompasses a whole domain. We decided to proceed with the latter.

On this day, we also researched the endpoints we would use for the project. During the later parts of the day, while a learning session was taking place, I exerted effort to figure out how to use the Zendesk API. Eventually, I successfully managed to figure it out and documented the process on Notion so that other team members could read through how to make requests to the Zendesk API.

Day 3 - Research continuation

Most of the non-coding work for the project was completed at this point, and we also theoretically established a way to test our application. Now, it's just up to the coding part.

We discussed the flow and logic for the application and created a flowchart to follow during implementation. Additionally, we made diagrams illustrating how we envisioned the deployment with Lambda and its behavior so that everyone understands what we are building and its components.

Once again, we decided to divide the work among the developers. Hemant, the senior developer in the group, was tasked with researching and setting up the deployment of the application. Meanwhile, Junsaku and I were assigned to implement the logic for the application and test it, ensuring it performs as intended without disrupting the ongoing tickets on production.

I delved further into Zendesk, trying out some APIs and double-checking what we needed. I documented my findings on a Notion page, which served as the common point for the hackathon project. Simultaneously, Hemant requested permissions for his AWS sandbox account from the tech leads. As we follow the practice of limiting access to certain services on AWS through IAM, we needed these permissions to access the AWS Lambda service.

In the afternoon, we didn't make much progress on our hackathon project as there were scheduled games and learning sessions that everyone onsite attended.

Day 4 - Implementation

On the 4th day, I walked from the hotel where we were staying to our company office, which is over 2 km away. I collaborated with Junsaku, guiding him on how to work with the Zendesk SDK and clarifying understandings and some questions. I outlined the prerequisites for him and set up a pull request to an existing Lambda project. After laying out the initial code for fetching the desired tickets on Zendesk, I immediately shared the information with him and paired with him on implementing the filtering and logic for the list of fetched Zendesk tickets. Throughout the process, I showed him how to create unit tests for the functions we wrote.

Unit tests played a significant role for us in this stage since we were dealing with production data. I cannot describe how nervous I was working with production initially. However, as we figured out how to create unit tests, we implemented and tested our code without accessing real production data. We simply mocked the responses and API calls in our tests, allowing us to test our code without the fear of disrupting production.

Although there were scheduled activities in the afternoon, unfortunately, my body temperature had risen a bit, and I experienced a headache due to lack of sleep. Consequently, I took the rest of the day off to rest.

In the evening, we had a scheduled team-building activity in Ayala IT Park. After somewhat feeling better, I rushed to join the others at the agreed venue for the team-building activity. I talked with Junsaku about the status of things, mentioning that everything had been implemented but still needed testing for multiple ticket updates. I suggested that we could address it the next day and proceed with deploying the project.

Day 5 - Demo day

It's the last day of the hackathon, and things are not looking well for us.

Hemant was still encountering issues setting up the Lambda for deployment. Despite the back-and-forths with the tech leads and all the trial and error that took a few hours, they hadn't figured out the necessary permissions for us to proceed.

I tested the code in the morning and confirmed that it was already working; we just needed to get it deployed. It was challenging for us to reach out to the tech leads because they had meetings scheduled throughout the week, and with Hemant joining online, the conversation became more asynchronous.

We needed to pivot to another way to demo our project. Ideally, the application should be triggered automatically every 15 minutes, the interval our team had agreed upon in our initial planning. I suggested that we extract the code into a new Spring Boot application and manually start it on a local environment to simulate the application trigger. We just needed to set up the Zendesk test tickets to have their scheduled work time within the desired time range. With that setup, we could fetch the tickets and perform update operations to reactivate them. It was clear at this point that we had missed showcasing the selling point of our project, which is the automation aspect of the process. However, we still had to proceed with the demo since we successfully built the logic needed for the tickets to be reactivated.

I communicated this information to the rest of the team about the state of things and the reason we were unable to deploy the project on AWS. We were going to rely on Harold and Queenie for the presentation, so we clearly explained the technical challenges that the we, the developers, had encountered.

We proceeded with the demo, setting up the test tickets on the spot. The test tickets didn't take too long to set up, so it was fine for the audience. We manually ran the application on a local environment, and it performed as we promised. We also explicitly stated to the audience that it was supposed to replace the current automation of the NOC team due to the limitations of the current automation. After the demo, we answered clarifications and questions from the rest of the audience. The tech leads also offered their apologies for not being able to resolve the issue with access to AWS Lambda after the presentation.

After all the demos were done, the awards were presented. The most-liked project received the People's Choice Award, determined via a vote by all participants of the hackathon. Unfortunately, our project didn't win the award, but every team member gave their all to make the project successful. Appreciation points were awarded to all participants of the hackathon, and after all the dust settled, all participants huddled in for a group photo to commemorate the event.

Takeaways

• Always prepare and never underestimate any situations:

  • I believe one of the areas that can certainly be improved in my second hackathon experience was the level of preparedness of our team. Although it wasn't my first time participating in a hackathon, it was my first time working with the people on my team. Consequently, we didn't communicate much until the week of the hackathon when it was very close.
  • Another aspect I underestimated was the challenges we would face during the deployment of the application. In theory, it seemed straightforward, and we were confident that we could finish testing and deployment within an hour if we had the necessary permissions. However, we underestimated the availability of the tech leads, resulting in us being unable to deploy our app on Lambda.

• Collaborations are more improved in physical interactions:

  • I am now convinced that teams working in person and in a face-to-face setting are likely to accomplish tasks faster than asynchronous teams.

• Team works best if everyone knows what they need to do and what they can do:

  • If team members are vocal and communicate well about the necessary tasks, and proactively initiate conversations to get efforts started, then other members are likely to be more proactive and show effort too.
  • Initiating conversations about needed tasks also provides members of the group an idea of the roles they're going to play. In our experience, everyone started picking out responsibilities as soon as the required tasks were defined.

• Challenging situations bring people out of their comfort zones but also foster learning and growth:

  • Sometimes an unpleasant way to go, but often effective. I took charge as the team member who would organize and keep things aligned so that every member of the team would not be left out. A role in the team I did not expect to find myself in, but still, I am grateful for the opportunity to handle a group, document, and act as a point person for the team.
  • This also meant that I did a lot of talking to other people during the duration of the hackathon, something that I don't normally do.
  • When our initial plan to deploy our hackathon project on Lambda failed, we quickly extracted our code into a new Spring Boot app that we could use for the demo. This was uncomfortable, given that we did this on the last day of the hackathon, just a few hours before the demo. However, our concept was simple, so we didn't encounter major issues during the pivot.

• Asking for help is not a sign of incompetence but rather the willingness to learn:

  • Many times, we sought advice and help from seniors on other teams regarding testing and developing our project to minimize the risk of disrupting production. Looking back, I'm glad that I asked for help and did not just rely on my judgment because seeking advice from others provided me with various perspectives and approaches to solving problems.

• There's always a learning curve when you're starting to learn something new:

  • Zendesk was a new concept to me during the hackathon, and I did not get to try it until we got access to the Rise NOC team's production. I thought it would take me just a day to figure things out, but it took me two days to truly understand. This was challenging for us developers because other teams were pitching projects related to our usual tech stack. The anxiety of dealing with production data is no joke, especially since we practice and maintain different environments and tread carefully when dealing with production line items. Accept that there's always a learning curve; you can estimate how long it'll take you to learn something, but you can't be sure.

Conclusion

The overall hackathon experience was a blast (except for the part where I got sick). I had fun collaborating and working with other developers, and I also got to know more of the new ones joining our department. There is a lot of learning that I gained from the event, and I also got to experience working in a role that is somewhat new to me. I am really grateful for my teammates for being such an awesome bunch. They really don't know how to give up, even if we were a little disheartened when we failed to deploy our project. I really appreciate the amount of effort, brainstorming, and collaboration that they put in to make our project successful. I also hope everyone had fun building and designing solutions for real-world problems that we face in our day-to-day work life. I am truly grateful for the opportunity to work with wonderful people.