best software to organize research papers

Your personal research assistant

Zotero is a free, easy-to-use tool to help you collect, organize, annotate, cite, and share research.

Available for Mac, Windows, Linux, and iOS

Just need to create a quick bibliography? Try ZoteroBib .

Meet Zotero.

Collect with a click..

Zotero automatically senses research as you browse the web. Need an article from JSTOR or a preprint from arXiv.org? A news story from the New York Times or a book from a library? Zotero has you covered, everywhere.

Organize your way.

Zotero helps you organize your research any way you want. You can sort items into collections and tag them with keywords. Or create saved searches that automatically fill with relevant materials as you work.

Cite in style.

Zotero instantly creates references and bibliographies for any text editor, and directly inside Word, LibreOffice, and Google Docs. With support for over 10,000 citation styles, you can format your work to match any style guide or publication.

Stay in sync.

Zotero can optionally synchronize your data across devices, keeping your files, notes, and bibliographic records seamlessly up to date. If you decide to sync, you can also always access your research from any web browser.

Collaborate freely.

Zotero lets you co-write a paper with a colleague, distribute course materials to students, or build a collaborative bibliography. You can share a Zotero library with as many people you like, at no cost.

Zotero is open source and developed by an independent, nonprofit organization that has no financial interest in your private information. With Zotero, you always stay in control of your own data.

Still not sure which program to use for your research? See why we think you should choose Zotero .

Ready to try Zotero?

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER COLUMN
• 07 July 2022

How to find, read and organize papers

• Maya Gosztyla 0

Maya Gosztyla is a PhD student in biomedical sciences at the University of California, San Diego.

You can also search for this author in PubMed   Google Scholar

“I’ll read that later,” I told myself as I added yet another paper to my 100+ open browser tabs.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

doi: https://doi.org/10.1038/d41586-022-01878-7

This is an article from the Nature Careers Community, a place for Nature readers to share their professional experiences and advice. Guest posts are encouraged .

Competing Interests

The author declares no competing interests.

Related Articles

• Research management

How to spot a predatory conference, and what science needs to do about them: a guide

Career Feature 30 JUL 24

Predatory conferences are on the rise. Here are five ways to tackle them

Editorial 30 JUL 24

We are junior scientists from emerging economies — the world needs more researchers like us solving global problems

Career Column 26 JUL 24

Sexual harassment in science: biologists in India speak out

News Feature 29 JUL 24

Why you should perform a premortem on your research

Career Column 24 JUL 24

So you got a null result. Will anyone publish it?

News Feature 24 JUL 24

A stroke derailed my academic career. Now I’m getting back on track

Career Column 23 JUL 24

Senior Research Assistant in Human Immunology (wet lab)

Senior Research Scientist in Human Immunology, high-dimensional (40+) cytometry, ICS and automated robotic platforms.

Boston, Massachusetts (US)

Boston University Atomic Lab

Postdoctoral Fellow in Systems Immunology (dry lab)

Postdoc in systems immunology with expertise in AI and data-driven approaches for deciphering human immune responses to vaccines and diseases.

Postdoctoral Fellow in Human Immunology (wet lab)

Join Atomic Lab in Boston as a postdoc in human immunology for universal flu vaccine research. Expertise in cytometry, cell sorting, scRNAseq.

Post-doctoral researcher (m/f/d) in ‘‘Seaweed polymer/biomolecule chemistry and valorisation‘‘

GEOMAR Helmholtz Centre for Ocean Research Kiel, Wischhofstraße 1-3, 24148 Kiel, Germany

Helmholtz Centre for Ocean Research Kiel (GEOMAR)

Faculty Positions at IISc Medical School

Indian Institute of Science (IISc), Bengaluru, India, IISc is India’s top-ranked research institution

Bangalore City, Bangalore (IN)

Indian Institute of Science (IISc)

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Top 21 must-have digital tools for researchers

Last updated

12 May 2023

Reviewed by

Jean Kaluza

Research drives many decisions across various industries, including:

Uncovering customer motivations and behaviors to design better products

Assessing whether a market exists for your product or service

Running clinical studies to develop a medical breakthrough

Conducting effective and shareable research can be a painstaking process. Manual processes are sluggish and archaic, and they can also be inaccurate. That’s where advanced online tools can help. 

The right tools can enable businesses to lean into research for better forecasting, planning, and more reliable decisions. 

• Why do researchers need research tools?

Research is challenging and time-consuming. Analyzing data , running focus groups , reading research papers , and looking for useful insights take plenty of heavy lifting. 

These days, researchers can’t just rely on manual processes. Instead, they’re using advanced tools that:

Speed up the research process

Enable new ways of reaching customers

Improve organization and accuracy

Allow better monitoring throughout the process

Enhance collaboration across key stakeholders

• The most important digital tools for researchers

Some tools can help at every stage, making researching simpler and faster.

They ensure accurate and efficient information collection, management, referencing, and analysis. 

Some of the most important digital tools for researchers include:

Research management tools

Research management can be a complex and challenging process. Some tools address the various challenges that arise when referencing and managing papers. 

.css-10ptwjf{-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;background:transparent;border:0;color:inherit;cursor:pointer;-webkit-flex-shrink:0;-ms-flex-negative:0;flex-shrink:0;-webkit-text-decoration:underline;text-decoration:underline;}.css-10ptwjf:disabled{opacity:0.6;pointer-events:none;} Zotero

Coined as a personal research assistant, Zotero is a tool that brings efficiency to the research process. Zotero helps researchers collect, organize, annotate, and share research easily. 

Zotero integrates with internet browsers, so researchers can easily save an article, publication, or research study on the platform for later. 

The tool also has an advanced organizing system to allow users to label, tag, and categorize information for faster insights and a seamless analysis process. 

Messy paper stacks––digital or physical––are a thing of the past with Paperpile. This reference management tool integrates with Google Docs, saving users time with citations and paper management. 

Referencing, researching, and gaining insights is much cleaner and more productive, as all papers are in the same place. Plus, it’s easier to find a paper when you need it. 

Acting as a single source of truth (SSOT), Dovetail houses research from the entire organization in a simple-to-use place. Researchers can use the all-in-one platform to collate and store data from interviews , forms, surveys , focus groups, and more. 

Dovetail helps users quickly categorize and analyze data to uncover truly actionable insights . This helps organizations bring customer insights into every decision for better forecasting, planning, and decision-making. 

Dovetail integrates with other helpful tools like ​Slack, Atlassian, Notion, and Zapier for a truly efficient workflow.

Putting together papers and referencing sources can be a huge time consumer. EndNote claims that researchers waste 200,000 hours per year formatting citations. 

To address the issue, the tool formats citations automatically––simultaneously creating a bibliography while the user writes. 

EndNote is also a cloud-based system that allows remote working, multiple-user interaction and collaboration, and seamless working on different devices. 

Information survey tools

Surveys are a common way to gain data from customers. These tools can make the process simpler and more cost-effective. 

With ready-made survey templates––to collect NPS data, customer effort scores , five-star surveys, and more––getting going with Delighted is straightforward. 

Delighted helps teams collect and analyze survey feedback without needing any technical knowledge. The templates are customizable, so you can align the content with your brand. That way, the survey feels like it’s coming from your company, not a third party. 

SurveyMonkey

With millions of customers worldwide, SurveyMonkey is another leader in online surveys. SurveyMonkey offers hundreds of templates that researchers can use to set up and deploy surveys quickly. 

Whether your survey is about team performance, hotel feedback, post-event feedback, or an employee exit, SurveyMonkey has a ready-to-use template. 

Typeform offers free templates you can quickly embed, which comes with a point of difference: It designs forms and surveys with people in mind, focusing on customer enjoyment. 

Typeform employs the ‘one question at a time’ method to keep engagement rates and completions high. It focuses on surveys that feel more like conversations than a list of questions.

Web data analysis tools

Collecting data can take time––especially technical information. Some tools make that process simpler. 

For those conducting clinical research, data collection can be incredibly time-consuming. Teamscope provides an online platform to collect and manage data simply and easily. 

Researchers and medical professionals often collect clinical data through paper forms or digital means. Those are too easy to lose, tricky to manage, and challenging to collaborate on. 

With Teamscope, you can easily collect, store, and electronically analyze data like patient-reported outcomes and surveys. 

Heap is a digital insights platform providing context on the entire customer journey . This helps businesses improve customer feedback , conversion rates, and loyalty. 

Through Heap, you can seamlessly view and analyze the customer journey across all platforms and touchpoints, whether through the app or website. 

Another analytics tool, Smartlook, combines quantitative and qualitative analytics into one platform. This helps organizations understand user behavior and make crucial improvements. 

Smartlook is useful for analyzing web pages, purchasing flows, and optimizing conversion rates. 

Project management tools

Managing multiple research projects across many teams can be complex and challenging. Project management tools can ease the burden on researchers. 

Visual productivity tool Trello helps research teams manage their projects more efficiently. Trello makes product tracking easier with:

A range of workflow options

Unique project board layouts

Advanced descriptions

Integrations

Trello also works as an SSOT to stay on top of projects and collaborate effectively as a team. 

To connect research, workflows, and teams, Airtable provides a clean interactive interface. 

With Airtable, it’s simple to place research projects in a list view, workstream, or road map to synthesize information and quickly collaborate. The Sync feature makes it easy to link all your research data to one place for faster action. 

For product teams, Asana gathers development, copywriting, design, research teams, and product managers in one space. 

As a task management platform, Asana offers all the expected features and more, including time-tracking and Jira integration. The platform offers reporting alongside data collection methods , so it’s a favorite for product teams in the tech space.

Grammar checker tools

Grammar tools ensure your research projects are professional and proofed. 

No one’s perfect, especially when it comes to spelling, punctuation, and grammar. That’s where Grammarly can help. 

Grammarly’s AI-powered platform reviews your content and corrects any mistakes. Through helpful integrations with other platforms––such as Gmail, Google Docs, Twitter, and LinkedIn––it’s simple to spellcheck as you go. 

Another helpful grammar tool is Trinka AI. Trinka is specifically for technical and academic styles of writing. It doesn’t just correct mistakes in spelling, punctuation, and grammar; it also offers explanations and additional information when errors show. 

Researchers can also use Trinka to enhance their writing and:

Align it with technical and academic styles

Improve areas like syntax and word choice

Discover relevant suggestions based on the content topic

Plagiarism checker tools

Avoiding plagiarism is crucial for the integrity of research. Using checker tools can ensure your work is original. 

Plagiarism checker Quetext uses DeepSearch™ technology to quickly sort through online content to search for signs of plagiarism. 

With color coding, annotations, and an overall score, it’s easy to identify conflict areas and fix them accordingly. 

Duplichecker

Another helpful plagiarism tool is Duplichecker, which scans pieces of content for issues. The service is free for content up to 1000 words, with paid options available after that. 

If plagiarism occurs, a percentage identifies how much is duplicate content. However, the interface is relatively basic, offering little additional information.  

Journal finder tools

Finding the right journals for your project can be challenging––especially with the plethora of inaccurate or predatory content online. Journal finder tools can solve this issue. 

Enago Journal Finder

The Enago Open Access Journal Finder sorts through online journals to verify their legitimacy. Through Engao, you can discover pre-vetted, high-quality journals through a validated journal index. 

Enago’s search tool also helps users find relevant journals for their subject matter, speeding up the research process. 

JournalFinder

JournalFinder is another journal tool that’s popular with academics and researchers. It makes the process of discovering relevant journals fast by leaning into a machine-learning algorithm.

This is useful for discovering key information and finding the right journals to publish and share your work in. 

Social networking for researchers

Collaboration between researchers can improve the accuracy and sharing of information. Promoting research findings can also be essential for public health, safety, and more. 

While typical social networks exist, some are specifically designed for academics.

ResearchGate

Networking platform ResearchGate encourages researchers to connect, collaborate, and share within the scientific community. With 20 million researchers on the platform, it's a popular choice. 

ResearchGate is founded on an intention to advance research. The platform provides topic pages for easy connection within a field of expertise and access to millions of publications to help users stay up to date. 

Academia is another commonly used platform that connects 220 million academics and researchers within their specialties. 

The platform aims to accelerate research with discovery tools and grow a researcher’s audience to promote their ideas. 

On Academia, users can access 47 million PDFs for free. They cover topics from mechanical engineering to applied economics and child psychology. 

• Expedited research with the power of tools

For researchers, finding data and information can be time-consuming and complex to manage. That’s where the power of tools comes in. 

Manual processes are slow, outdated, and have a larger potential for inaccuracies. 

Leaning into tools can help researchers speed up their processes, conduct efficient research, boost their accuracy, and share their work effectively. 

With tools available for project and data management, web data collection, and journal finding, researchers have plenty of assistance at their disposal.

When it comes to connecting with customers, advanced tools boost customer connection while continually bringing their needs and wants into products and services.

What are primary research tools?

Primary research is data and information that you collect firsthand through surveys, customer interviews, or focus groups. 

Secondary research is data and information from other sources, such as journals, research bodies, or online content. 

Primary researcher tools use methods like surveys and customer interviews. You can use these tools to collect, store, or manage information effectively and uncover more accurate insights. 

What is the difference between tools and methods in research?

Research methods relate to how researchers gather information and data. 

For example, surveys, focus groups, customer interviews, and A/B testing are research methods that gather information. 

On the other hand, tools assist areas of research. Researchers may use tools to more efficiently gather data, store data securely, or uncover insights. 

Tools can improve research methods, ensuring efficiency and accuracy while reducing complexity.

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 18 April 2023

Last updated: 27 February 2023

Last updated: 6 February 2023

Last updated: 6 October 2023

Last updated: 5 February 2023

Last updated: 16 April 2023

Last updated: 9 March 2023

Last updated: 12 December 2023

Last updated: 11 March 2024

Last updated: 4 July 2024

Last updated: 6 March 2024

Last updated: 5 March 2024

Last updated: 13 May 2024

Latest articles

Related topics, .css-je19u9{-webkit-align-items:flex-end;-webkit-box-align:flex-end;-ms-flex-align:flex-end;align-items:flex-end;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;row-gap:0;text-align:center;max-width:671px;}@media (max-width: 1079px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}}@media (max-width: 799px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}} decide what to .css-1kiodld{max-height:56px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}@media (max-width: 1079px){.css-1kiodld{display:none;}} build next, decide what to build next.

Users report unexpectedly high data usage, especially during streaming sessions.

Users find it hard to navigate from the home page to relevant playlists in the app.

It would be great to have a sleep timer feature, especially for bedtime listening.

I need better filters to find the songs or artists I’m looking for.

Log in or sign up

Get started for free

30+ Essential Software for Researchers

Are you stuck with inefficient research tools? Discover the best essential software for researchers to revolutionize your work.

But here’s the good news: online tools for researchers can turn these challenges into manageable tasks.

Table of Contents

This paradigm shift signifies the evolution of academia from a space where only the intellectually elite thrive, to a nurturing environment that promotes intellectual curiosity and learning for all.

Essential Software for Researchers

#1. google scholar: best for scholarly literature search and keeping up-to-date with research in your field.

It helps you pinpoint where your investigation could contribute to the existing body of knowledge. Here are other academic journal discovery platforms that can help you at this stage of research .

#2. NVivo: Best for Designing and Conducting Qualitative Research

NVivo stands as a beacon of hope for qualitative researchers in the data fog. Its unique features categorize, analyze, and draw connections like a seasoned detective, unearthing meaningful insights with ease. 

Source: Lumivero

#3. Qualtrics: Best for Survey Design and Distribution

In the world of survey research, Qualtrics is your steadfast ally. It not only simplifies the process but also empowers you to glean meaningful insights from the data, adding immense value to your research. You can also check out other survey tools.

#4. SPSS: Best for Statistical Analysis and Data Interpretation

If statistical analysis is your battleground, SPSS becomes your formidable armor. This software doesn’t merely crunch numbers; it possesses the alchemical ability to transform them into comprehensible insights, making data interpretation a breeze rather than a battle. 

Source: https://www.ibm.com

#5. Tableau: Best for Data Visualization and Reporting

How much does it cost?

#6. Overleaf: Best for Collaborative Writing and LaTeX Editing

Overleaf facilitates collaboration and simplifying the editing process, making crafting complex documents less daunting and more productive. You can learn more about LaTeX tutorials here.

#7. Grammarly: Best for Checking Grammar and Improving Writing Clarity

With Grammarly at your side, you’re not just writing; you’re crafting compelling narratives. This tool helps ensure that your ideas shine brightly, unmarred by grammatical errors or unclear writing.

#8. Turnitin: Best for Plagiarism Checking and Originality Reports

With Turnitin’s cutting-edge technology, students and educators can have the confidence that their academic pursuits maintain the highest standards of integrity and authenticity. 

#9. Mendeley: Best for Discovering New Research and Collaborative Work

Source: https://www.mendeley.com

#10. Zotero: Best for Collecting, Organizing, and Citing Research Sources

Source: https://www.zotero.org

#11. Trello: Best for Research Project Management and Task Organization

#12. researchgate: best for connecting with fellow researchers and sharing publications.

This dynamic environment empowers you to stay at the forefront of knowledge and contribute to the scientific community.

#13. Notion: Best for Comprehensive Note-Taking and Project Management

With Notion, the tedious becomes straightforward, the overwhelming becomes manageable, and the complex becomes clear. It’s about getting the most out of your A-level studies, fostering a sense of achievement while making the process enjoyable. So, buckle up and let Notion revolutionize the way you work.

Source: https://www.notion.so

#14. Quillbot: Best for Paraphrasing and Improving Writing Clarity

Here are other academic writing tools you may need.

#15. Jasper AI: Best for AI-Powered Writing Assistance

Ever dreamt of having a personal writing mentor, constantly at your beck and call, simplifying the intricacies of academic writing for you? Welcome Jasper AI into your world – an exemplary writing companion that surpasses the functionalities of a typical digital assistant. 

#16. GanttPRO: Best for Project Scheduling and Time Management

Source: https://ganttpro.com

#17. Scholarcy: Best for Quick Summarization of Academic Papers

Source: https://www.scholarcy.com

#18. R Discovery: Best for Statistical Analysis and Data Visualization

Source: https://discovery.researcher.life

#19. Scopus: Best for Comprehensive Literature Search and Citation Tracking

Source: https://www.scopus.com

#20. Journal Finder: Best for Identifying the Right Journals for Publishing Your Research

Journal Finder serves as your publishing compass, steering you towards the right journals to publish your research. This tool saves you from the guesswork, maximizing the chances of your work reaching the right audience.

Source: https://journalfinder.elsevier.com

#21. Global Journal Database: Best for Accessing Information about Various Journals

Source: https://researcher.life

#22. Citation Gecko: Best for Literature Review and Citation Network Exploration

Source: https://www.citationgecko.com

#23. OpenRefine: Best for Cleaning and Transforming Messy Data

Source: https://openrefine.org

#24. MATLAB: Best for Complex Mathematical Calculations and Data Analysis

Source: https://www.mathworks.com

#25. Amazon Drive: Best for Storing and Sharing Research Files

Source: https://www.amazon.com

#26. Otter.ai: Best for Transcription of Interviews and Meetings

Source: https://otter.ai

#27. LabView: Best for Data Acquisition and Instrument Control in Lab Environments

LabView is not just a lab tool—it’s a catalyst for efficiency and precision. By facilitating data acquisition, instrument control, and real-time analysis, it turns your lab into a hub of productivity, taking your research a notch higher.

Source: https://www.ni.com

#28. SAS: Best for Advanced Statistical Analysis and Predictive Modeling

Source: https://www.sas.com

#29. BioRender: Best for Creating Scientific Figures and Illustrations

BioRender is not just a graphics tool—it’s a bridge between your research and your audience. It aids in communicating your findings more effectively, amplifying the impact of your work.

Source: https://www.biorender.com

#30. Slack: Best for Team Communication and Collaboration

Source: https://slack.com

#31. RStudio: Best for Statistical Computing and Graphics in R

RStudio is your personal statistician, providing a comprehensive environment for R, a popular language for statistical computing. 

Academic research isn’t just about the pursuit of knowledge; it’s about leveraging the right tools to streamline that pursuit. As we’ve explored, these essential research software applications aren’t merely aids. 

Leave a Comment Cancel reply

12 Best Software for Organizing Research (2024)

Discover our guide with the best software for organizing research; get on top of your academic papers, writing projects, and more with these top research organization assistants.

Whether researching for an academic paper, independent study or writing project, having an organization system is essential for managing your data. The right software can help to save time, promote teamwork and increase productivity, no matter the type of project.

Often, when starting a new writing project, like writing a historical fiction piece, I’ll use writing software to collate all of my data. Organizing my research on a digital platform lets me visualize all the information and quickly find facts and figures. Check out our writing tips for your next project.

Research organization tools are excellent for academic writing and helpful for students getting their heads around how to structure data correctly. Let’s check out why you should organize your research and the best software to do it with.

Why Organize Research Data?

Key features of research software, 2. mendeley, 3. evernote, academic writing and editing solutions, 4. grammarly, 5. hemingway editor, 6. prowritingaid, collaborative research platforms, 7. google docs, 8. authorea, 9. microsoft teams, data extraction and analysis tools, 10. octoparse, 11. contentmine, 12. tableau, what is an excellent way to organize research information, how does research organization software improve the quality of academic work.

• It saves time. Instead of searching for information, you can focus on analyzing it.
• Better collaboration. With software, teamwork becomes more straightforward, no matter where your team members are.
• Improved work quality. Software ensures your data is well-organized and easy to access, enhancing the accuracy of your results.
• Time Management: Software can automate repetitive tasks, helping researchers be more efficient.
• Collaboration: Working together is essential in research. Software tools make it easy to communicate and ensure everyone is on the same page.
• Quality: Software helps maintain high-quality work by organizing data and providing easy access.

Zotero is a personal research assistant who collects and organizes your research data on one platform. Not only does Zotero sort and organize your data, but it also assists with citations, which can be one of the research’s trickiest parts.

Zotero is ideal for academic writing, where you will need to create a bibliography. With an intuitive interface and helpful tools, Zotero can help you create seamless academic documents without having to tidy up your research sections.

Imagine having a virtual library at your fingertips, with all your research sources meticulously organized, including your pdf files and web pages – that’s the power of Mendeley ! Mendeley is a reference manager suited well to academic writing.

Mendeley makes writing papers less daunting, from facilitating collaboration to offering annotation and data extraction features. Despite some limitations, such as limited customization options, Mendeley shines as a tool that simplifies library management for researchers.

Evernote is a go-to note-taking software and an excellent research organization assistant. Evernote has a user-friendly interface that’s easy to navigate. With synchronization capabilities and intuitive tools, you can quickly jot down ideas and organize your research notes. A free version of the software is available; although it’s limited, it’s an excellent solution for students on a budget.

Academic writing is an art. It demands precision, clarity, and comprehensive knowledge of the subject matter. But even the most experienced writers need help to ensure their work is up to par.

Grammarly is like a personal English tutor, constantly guiding your writing towards precision and clarity. Grammarly is ideal for organizing your research because it can help create concise notes and format citations, and its new AI, GrammarlyGO, can even help generate text. 

Although it’s not the best for presenting your research data, it’s a secure online platform where you can save your research and citations for easy access from anywhere. Check out our guide on how to organize in-text citations .

Writing is about communicating ideas clearly and simply. Hemingway Editor is designed to help you create clear and concise research notes. This tool is helpful for academic students looking to condense and clarify their research notes, particularly after conducting qualitative research like interviews.

The Hemingway Editor acts as a writing coach, pointing out long-winded sentences, unnecessary words, and complex phrases. It helps you simplify your writing, ensuring your ideas are communicated effectively. 

ProWritingAid is an all-in-one writing assistant that offers comprehensive writing assistance. It can analyze text for clarity, grammar, style and structure, ensuring that the information derived from your research is clear.

Effectively communicating your research findings is essential for academic success. To create a valuable research document, refine your research and organize it within ProWritingAid. Although this software doesn’t directly organize research, it’s a great way to enhance the quality of your content.

Research is often a team effort. And in today’s digital age, collaboration isn’t bound by geographical boundaries. Collaborative research platforms bridge the gap, providing a virtual space for researchers to work together. What is academic writing? Check out our guide to find out.

GoogleDocs is ideal for collaborative research organizations. Teams can collaborate on one document that updates in real time, providing a secure and functional space for collaboration. Using GoogleDocs, team members can add research data, tables, charts or statistical data, which the team can then work on.

As well as the functionality of the data presenting tools, GoogleDocs lets users view previous versions of the document. This essential feature ensures that no data is lost permanently and can be retrieved in the case of an accident.

Authorea is specifically designed for researchers. This software is a collaborative writing tool that allows authors to make real-time changes. Authorea stands out as a research organization tool because it has features like creating scientific notations, figures and data visualizations. As well as this, Authorea integrates with bibliographic tools, allowing users to create reference pages and bibliographies easily. This tool is most suited to students and academics.

Communication is key in Research. From discussing ideas to sharing findings, efficient communication ensures everyone is on the same page. Microsoft Teams provides a centralized platform for research communication, bridging the gap between team members.

Microsoft Teams is like a virtual office, providing a space for discussions, meetings, and information sharing. Features like document collaboration, video conferencing, and team chats ensure seamless and efficient communication. While it may initially seem overwhelming, Microsoft Teams is a robust platform that can significantly enhance research communication.

Data is the lifeblood of Research. But collecting and analyzing all the data can be a daunting task. Thankfully, there are tools to make this process easier. Data extraction and analysis tools simplify the process, allowing researchers to focus on what matters – the Research.

Octoparse simplifies this process of extracting data from websites. It’s like having a personal data miner digging through the vast internet landscape to find the necessary data.

With Octoparse, you can automate the web scraping process, saving time and effort. It’s like using a metal detector to find that elusive needle in the haystack. While web scraping may seem complex, Octoparse makes it accessible to everyone, regardless of their technical skills.

Academic papers, including research and scientific papers, are a goldmine of information. However, extracting this information can take time and effort. ContentMine is like a virtual miner, digging through academic papers to extract valuable information.

With ContentMine, you can automate extracting data from academic papers. It’s like having a speed reader who can quickly scan through documents and extract the information you need. While text mining may seem daunting, ContentMine makes it accessible and efficient.

Tableau is a powerful tool for data visualization that can help bring your research data to life! Using Tableau, you can make your data easy to understand for all audiences. It’s like having a personal graphics designer who can transform raw data into a visually pleasing infographic. While data visualization may seem complex, Tableau makes it easy and accessible to everyone.

FAQs About Best Software for Organizing Research

Organize research information by tracking search processes using a research log or spreadsheet and use citation managers to create bibliographies. Additionally, it utilizes coloring and tagging techniques to assign meaning to data during the synthesis process.

Research organization software helps maintain the integrity and quality of academic work by providing a structured approach to data storage and accessibility. This ensures that information is readily available, reducing errors that arise from misplaced or misinterpreted data. 

Looking for more? Check out our round-up of the best essay writing apps !

• Live Stream the Summer Olympics
• Killer Laptop Deals Available Now

15 Best Free Web Tools to Organize Your Research

How to stay organized when researching and writing papers

• Emporia State University
• Cloud Services
• Error Messages
• Family Tech
• Home Networking
• Around the Web

Organizing research is important not only for your own sanity, but because when it comes time to unfold the data and put it to use, you want the process to go as smoothly as possible. This is where research organizers come in.

There are lots of free web-based organizers that you can use for any purpose. Maybe you're collecting interviews for a news story, digging up newspaper archives for a history project, or writing a research paper over a science topic. Research organizers are also helpful for staying productive and preparing for tests.

Regardless of the topic, when you have multiple sources of information and lots to comb through later, optimizing your workflow with a dedicated organizer is essential.

Patrick Tomasso / Unsplash

Many of these tools provide unique features, so you might decide to use multiple resources simultaneously in whatever way suits your particular needs.

Research and Study

You need a place to gather the information you're finding. To avoid a cluttered space when collecting and organizing data, you can use a tool dedicated to research.

• Pocket : Save web pages to your online account to reference them again later. It's much tidier than bookmarks, and it can all be retrieved from the web or the Pocket mobile app .
• Mendeley : Organize papers and references, and generate citations and bibliographies.
• Quizlet : Learn vocabulary with these free online flashcards .
• Wikipedia : Find information on millions of different topics.
• Quora : This is a question and answer website where you can ask the community for help with any question.
• SparkNotes : Free online study guides on a wide variety of subjects, anything from famous literary works of the past century to the present day. 
• Zotero : Collect, manage, and cite your research sources. Lets you organize data into collections and search through them by adding tags to every source. This is a computer program, but there's a browser extension that helps you send data to it.
• Google Scholar : A simple way to search for scholarly literature on any subject.
• Diigo : Collect, share, and interact with information from anywhere on the web. It's all accessible through the browser extension and saved to your online account.
• GoConqr : Create flashcards, mind maps, notes, quizzes, and more to bridge the gap between your research and studying.

Writing Tools

Writing is the other half of a research paper, so you need somewhere useful to go to jot down notes, record information you might use in the final paper, create drafts, track sources, and finalize the paper.

• Web Page Sticky Notes : For Chrome users, this tool lets you place sticky notes on any web page as you do your research. There are tons of settings you can customize, they're backed up to your Google Drive account, and they're visible not only on each page you created them on but also on a single page from the extension's settings.
• Google Docs or Word Online : These are online word processors where you can write the entire research paper, organize lists, paste URLs, store off-hand notes, and more.
• Google Keep : This note-taking app and website catalogs notes within labels that make sense for your research. Access them from the web on any computer or from your mobile device. It supports collaborations, custom colors, images, drawings, and reminders.
• Yahoo Notepad : If you use Yahoo Mail , the notes area of your account is a great place to store text-based snippets for easy recall when you need them.
• Notion : Workflows, notes, and more, in a space where you can collaborate with others.

Get the Latest Tech News Delivered Every Day

• 10 Free Image Hosting Sites for Your Photos
• The Best Free People Search Websites
• The Best Research and Reference Websites
• The Best Free Foreclosure Search Sites
• The 10 Best Bookmarking Tools for the Web
• 16 Best Ways to Get Free Textbooks
• How to Use the Invisible Web to Find People
• 5 Best Ways to Find a Cell Phone Number Online
• The 8 Best Ways to Send Large Files Over the Internet
• What Is a Web Directory?
• Micromobility World: Dates, News, Rumors, and Everything Else To Know
• 5 Best Translation Sites of 2024
• Best Niche Search Engines
• 4 of Our Favorite Free Online Calendars
• The Best Image Search Engines on the Web
• Internet vs. Web: What's the Difference?

Something went wrong when searching for seed articles. Please try again soon.

No articles were found for that search term.

Author, year The title of the article goes here

LITERATURE REVIEW SOFTWARE FOR BETTER RESEARCH

“Litmaps is a game changer for finding novel literature... it has been invaluable for my productivity.... I also got my PhD student to use it and they also found it invaluable, finding several gaps they missed”

Varun Venkatesh

Austin Health, Australia

As a full-time researcher, Litmaps has become an indispensable tool in my arsenal. The Seed Maps and Discover features of Litmaps have transformed my literature review process, streamlining the identification of key citations while revealing previously overlooked relevant literature, ensuring no crucial connection goes unnoticed. A true game-changer indeed!

Ritwik Pandey

Doctoral Research Scholar – Sri Sathya Sai Institute of Higher Learning

Using Litmaps for my research papers has significantly improved my workflow. Typically, I start with a single paper related to my topic. Whenever I find an interesting work, I add it to my search. From there, I can quickly cover my entire Related Work section.

David Fischer

Research Associate – University of Applied Sciences Kempten

“It's nice to get a quick overview of related literature. Really easy to use, and it helps getting on top of the often complicated structures of referencing”

Christoph Ludwig

Technische Universität Dresden, Germany

“This has helped me so much in researching the literature. Currently, I am beginning to investigate new fields and this has helped me hugely”

Aran Warren

Canterbury University, NZ

“I can’t live without you anymore! I also recommend you to my students.”

Professor at The Chinese University of Hong Kong

“Seeing my literature list as a network enhances my thinking process!”

Katholieke Universiteit Leuven, Belgium

“Incredibly useful tool to get to know more literature, and to gain insight in existing research”

KU Leuven, Belgium

“As a student just venturing into the world of lit reviews, this is a tool that is outstanding and helping me find deeper results for my work.”

Franklin Jeffers

South Oregon University, USA

“Any researcher could use it! The paper recommendations are great for anyone and everyone”

Swansea University, Wales

“This tool really helped me to create good bibtex references for my research papers”

Ali Mohammed-Djafari

Director of Research at LSS-CNRS, France

“Litmaps is extremely helpful with my research. It helps me organize each one of my projects and see how they relate to each other, as well as to keep up to date on publications done in my field”

Daniel Fuller

Clarkson University, USA

As a person who is an early researcher and identifies as dyslexic, I can say that having research articles laid out in the date vs cite graph format is much more approachable than looking at a standard database interface. I feel that the maps Litmaps offers lower the barrier of entry for researchers by giving them the connections between articles spaced out visually. This helps me orientate where a paper is in the history of a field. Thus, new researchers can look at one of Litmap's "seed maps" and have the same information as hours of digging through a database.

Baylor Fain

Postdoctoral Associate – University of Florida

We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.

Top 13 Tools for Researchers in 2024!

Gone are the days of going to the library, studying numerous books, taking notes on paper, and doing research. Thanks to technology, we no longer have to do this tedious routine to do research. However, research is still a meticulous, painstaking process.

This is why we decided to uncover some of the best software tools for researchers that are going to help you conduct and maintain your research with ease. Read on…

List of Top 13 Best Tools for Researchers for better results:

Research today is dynamic. We often use the internet to browse websites, watch videos, study analytics, and conduct our research by exploring different types of digital content, making technology a major stakeholder in making our research success .

While the internet has made it easy for us to access worldly information with the click of a button (or mouse!), it has created a whole new set of problems.

Sorting through a seemingly infinite number of websites, verifying content, and curating only the best stuff can take a lot of time and effort. This is why we have brought you 13 essential research tools every researcher should use while working on the internet.

1. Bit.ai  

Online research means going through numerous websites, articles, blogs, images, videos, infographics, and more to find what you are looking for.

For our dynamic, interactive, and media-rich research, we need a tool that incorporates all facets of modern-day research under one roof. Simple text editors of the past just won’t cut it anymore! This is where Bit comes in.

Bit allows researchers and teams to collaborate, share, track, and manage all knowledge and research in one place.

It’s the perfect research tool to share multi-dimensional research with your peers and not just plain, boring text and slides.

Add articles, PDFs, videos, white papers, ebooks, audio samples- basically anything you can think of – and share it with your peers easily!

Other notable features of Bit include:

• An easy-to-use, minimal editor that supports Markdown.
• Collaborative, real-time editing, and communication with peers.
• Add any type of digital content (images, videos, etc) to your Bit document.
• A content library to save all your media files for quick access.
• Smart search, allows anyone to search and find any files, images, documents, links, etc quickly.

All-in-all, Bit is a must-have writing tool for researchers and authors!

Key Features of Bit: 

• Workspaces to store different research content easily
• Content library to store media assets
• Real-time collaboration with fellow researchers
• Free with limited functionality
• Paid plans start from Pro ($8/month), Business ($15/month), Enterprise (contact sales)

Read more:  How Bit.ai Can Help You Manage Your Academic Research?

2. elink.io

Research often involves going through hundreds of links and articles and compiling them in one safe space for future reference or publishing them for your audience.

This is why many researchers use bookmarking and curation tools like elink to quickly save their links under one roof and share them with their peers.

elink makes it easy for researchers to save content from around the web. They can save article links, videos, cloud files, social media posts, and much more!

Researchers have the option of saving content to their link library or adding them directly to content collections and sharing their research with their peers . To make the bookmarking process a breeze, elink also has a chrome extension .

Simply click on the extension or right-click on any webpage to save the content directly to your elink dashboard.

Researchers can edit the title and description to add their own voices or notes. They can even bundle links together and share their link collection with others as a newsletter or embed the collection on your blog/website!

Key Features of elink:

• Save links quickly using the chrome extension
• Create and share research links as a newsletter or embed it on your website
• Easy user-interface
• Paid plans start at Pro Monthly ($15/month), Pro 1 Year ($12/month), and Pro 2 years ($10/month).

3. GanttPRO

No matter what kind of research you do, you need to organize, plan, and stay focused on all of your activities.

Without a robust planning tool, researchers may fall behind the schedule and lose their progress.

GanttPRO project and task management tool makes it easy for single researchers and groups of any size to plan their tasks on a visually appealing Gantt chart timeline, follow their progress, and all the deadlines.

GanttPRO allows researchers to create a limitless number of tasks, groups of tasks, and subtasks on one timeline.

Besides, it’s a perfect planning tool for assigning tasks to your fellow researchers or creating virtual resources, whoever or whatever they may be. The software is a good choice for collaboration, time tracking, as well as sharing and exporting your schedules.

Key Features of GanttPRO:

• Dozens of ready-made templates.
• Real-time collaboration with fellow researchers.
• Elegant user interface with a short learning curve.
•   Free 14-day trial with all features available.
• Paid plans start from Team ($4.5/user/month), Individual ($15/month), Enterprise (contact sales).

4. Grammarly

Research work often involves hours of proofreading and spellchecking to make your research professional .

Grammarly, a writing enhancement tool will save you a ton of time and effort doing this dreaded task! Apart from basic spellchecking and corrections, Grammarly includes a grammar checker, a punctuation checker, a vocabulary enhancer, and even a plagiarism checker tool!

This awesome tool scans your research for more than 250 types of grammar mistakes in six distinct writing genres and leaves you with error-free writing. With thorough explanations for all your errors and weekly progress reports .

Grammarly is a must-have tool for researchers. It’s available as a browser extension, a desktop app, a web-based app, and a Microsoft add-in. Many of the Grammarly alternatives are also available in the market that is equally good.

Key Features of Grammarly: 

• Works with the majority of online tools like Word, Slack, etc.
• Plagiarism checker tool
• Tone detector
• Paid plans start from: Premium ($11.66/month), Business ($12.50/month)

Read more:   10 Best Writing Apps To Make You A Better Writer!

5. Typeset.io

With over 100,000+ verified journal formats to choose from, Typeform makes the process of research a bit too easy! Quickly copy-paste or upload your paper on Typeset and follow any citation style you need.

Typeset also has a plagiarism and grammar checker built in to ensure your writing is error-free. Once done uploading and citing, click on autoformat to generate your report in seconds.

You can also download your research in PDF , Docx, LaTeX file, or even as a Zip file. With collaboration features built-in, you can invite your fellow researchers to the platform and work together.

Key Features of Typeset: 

• Over 100,000+ journal formats to choose from
• Plagiarism and grammar checker tool
• Editing services to improve your publication chances
• Paid plans start from: Researcher ($8/month), Team($6/month), Journals / Publishers (contact sales)

6. Scrivener

Scrivener is another great tool for research writing and keeping your notes organized.

Used by researchers, screenwriters, novelists, non-fiction writers, students, journalists, academics, lawyers, translators, and more, Scrivener is a tool made for long writing projects.

On signing up, you are quickly presented with its editor, with a sidebar to keep everything in place. You can also break your content into manageable sections of any size and leave Scrivener to join them together.

For novelists and storytellers, there’s also a corkboard to visualize your storyline and move cards around as you like.

The outliner keeps a synopsis of what you have already written, along with word count data and metadata. Users can arrange their research articles and other files in folders and subfolders.

Key Features of Scrivener: 

• Desktop and mobile apps
• Outline creator
• Easy organization
• Paid plans start from $40.84/one-time fee

7. ProofHub

You must organize, prepare, and stay focused on all of your efforts, regardless of the type of research you conduct.

Researchers may go behind schedule and lose progress if they don’t have a good task management tool. ProofHub is an all-in-one project and team management application that allows research teams and organizations of any size to efficiently plan their research projects in one spot.

ProofHub allows you to create, assign and track tasks using effective task management features like Kanban boards and table view. Researchers can also get a visual idea of how their project is progressing using robust Gantt charts.

ProofHub also allows you to store and jot down all the data or information collected through your research in Notes. You can even create different notebooks and store your information according to the topic. Not just that, you can even share your research work with your team members.

Teams can also share and store files, documents, and images in ProofHub’s files section. Managers can track their team’s time spent on a specific research task using automatic and manual timers.

Team members can also brainstorm ideas or have real-time discussions in ProofHub’s discussions section and make way for better research work. 

Key features of ProofHub:

• Ready to use project templates
• Task management
• Time tracking and project reporting
• Team collaboration (chat, notes, and discussions)
• File management
• Online proofing
• 14-day free trial with all the features.

8. Google Scholar

Next up is an amazing research tool by Google called Google Scholar. Google Scholar provides a quick way to broadly search for scholarly literature from one location.

Look for articles, theses, books, abstracts, and court opinions, from professional societies, online repositories, universities, academic publishers, and other websites.

Researchers can also explore related works, citations, authors, and publications easily. Create a public author profile and see who’s citing your recent publication. Google Scholar also allows its users to keep up with recent developments in any area of research.

Key Features of Google Scholar: 

• Create a public author page
• Look for information across Google’s database
• Easy to use
• Free to use

Endnote wants you to research smarter by simplifying the tiresome work of formatting bibliographies, finding full text, and searching for references.

Endnote is collaborative in nature as it allows you to share selected groups of references, manage team access, and track activity and changes from one single dashboard.

With smarter insights, Endnote automatically finds the impact of your references and finds the best-fit journal for your papers.

The platform also enables users to automatically create, format, and update bibliographies. Quickly export your references and full-text PDFs into EndNote and start working instantly.

With a bunch of EndNote templates and plug-ins, researchers can enhance their Endnote experience and get the most out of the platform.

Key Features of Endnote: 

• Import filters for prior research
• Track your teammates’ activity on your shared library
• Automatic reference and link updating
• Paid plans start from $249

10. Evernote

Evernote is a note-taking app that can be very useful while conducting research . The app helps you store all your personal ideas, to-do listsm4, notes, and research links in one place.

Create separate tags and folders for the different types of information you are saving and keep it all organized.

Evernote auto-syncs across all your devices, including desktop, smartphone, and tablet, so you can switch between devices without losing your data.

Its Chrome browser extension called the Evernote web clipper is a great add-on for saving articles or other content on the internet while doing your research.

Just click the browser extension to save the entire page or highlights to your Evernote notebook along with any notes you have about that page.

Key Features of Evernote: 

• Keep notes, articles, and other content in one place
• Chrome extension for clipping content
• Set reminders
• Paid plans start from Plus ($34.99 per year or $3.99 per month), Premium ($69.99 per year or $7.99 per month), and Evernote Business (contact sales)

11. Mendeley

Mendeley is a reference management software that allows researchers to create references, citations, and bibliographies in multiple journal styles with just a few clicks.

Quickly access your library from anywhere – from anywhere. Windows, Mac, Linux, etc and add papers directly from your browser with a few clicks or import any documents from your desktop to your library.

With its research network, researchers connect and network with over 6 million users. Users can create groups to carry out discussions, discover research, and follow curated bibliographies.

There are also over 250,000 + science, technology, and health jobs to advance your career and grant info from over 5000 organizations to fund your next research !

Key Features of Mendeley: 

• Annotate and organize documents
• Find and create groups with fellow researchers
• Grant information from over 5000 organizations
• Paid plans start from $55/year for 5 GB to $165/year to unlimited storage

12. ContentMine

ContentMine offers a variety of text mining services to help researchers find, download, analyze, and extract knowledge from academic papers.

ContentMine builds its own open-source code to help out researchers find papers and not waste time on the internet doing so. They can also convert academic papers , PDFs to HTML, or to almost any format.

ContentMine can also extract data from tables and graphs, reducing the time taken to conduct a meta-analysis. The platform also offers consultancy as well as training workshops to educate people on the work they do and how.

Key Features of ContentMine: 

• Extract data from tables and graphs
• Quickly mine text from hundreds of papers
• Workshops and training
• Contact sales

13. ResearchGate

The last tool on our list of awesome tools for researchers is a platform called ResearchGate. ResearchGate gives you access to over 135 million publication pages, allowing you to stay up to date with what’s happening in your field.

With a built-in community, researchers can share their research, collaborate with peers, and discover new papers and bibliographies.

ResearchGate also provides deep analytics on who’s been reading your work and keeps track of your citations. With over 17 million users, ResearchGate is a research community to join!

Key Features of ResearchGate: 

• Share and find researchers
• Analytics to see who’s reading your work
• Citation tracking

Before you go!

Our team at  bit.ai  has created a few awesome templates to make your research process more efficient. Make sure to check them out before you go, y our team might need them!

• Case Study Template
• Research Paper Template
• Competitor Research Template
• Brainstorming Template
• SWOT Analysis Template
• White Paper Template

Final Words

There you have it folks, our list of amazing websites, apps, and software to use while conducting your research. Research is hard work- from finding and managing content to organizing and publishing- research takes a lot of time and effort.

However, with our awesome list of tools, researchers are surely going to get out the most of their time and effort and get work done more efficiently. Did we miss any awesome tool for researchers out there? Let us know by tweeting us at @bit_docs.

Further reads:

• Top 11 Code Editors for Software Developers
• Collaborative Research: Definition, Benefits & Tips!
• Best Resource Management Tools and Software
• How to Write a Research Proposal?

11 Best Link in Bio Tools in 2024!

Vision Statements: Definitions, Examples, and Best Practices

Related posts

13 social media post ideas for interesting content, value proposition: what is it & how to write it, 9 virtual team building games for remote employees, 9 daily planners you can use to improve your productivity, short-form video content that are trending in 2024, best file management systems and software in 2024.

About Bit.ai

Bit.ai is the essential next-gen workplace and document collaboration platform. that helps teams share knowledge by connecting any type of digital content. With this intuitive, cloud-based solution, anyone can work visually and collaborate in real-time while creating internal notes, team projects, knowledge bases, client-facing content, and more.

The smartest online Google Docs and Word alternative, Bit.ai is used in over 100 countries by professionals everywhere, from IT teams creating internal documentation and knowledge bases, to sales and marketing teams sharing client materials and client portals.

👉👉Click Here to Check out Bit.ai.

Recent Posts

2024’s ultimate document template guide for modern educators, 20 ready-to-use marketing templates to elevate your business impact, top knowledge-sharing practices to maximize manufacturing performance, leverage the power of knowledge management to streamline manufacturing, top 20 user manual templates: a guide to their uses and applications, 20 top-rated customer service templates for outstanding support.

Educational resources and simple solutions for your research journey

5 Best Apps for Researchers: Apps that Every Researcher Should Know About

Today, one need not physically visit a library and take down notes on sheaves of paper (which are apt to fly about or clutter one’s workspace!). Various online tools and software applications (apps) have made our lives easier, especially the many helpful apps for researchers.

The use of apps to ease workload, manage time, or spark creativity are rapidly becoming de rigueur in all areas of work. In academia too, several apps for researchers are designed to help with daily activities, such as collecting and organizing resources, managing collaborative projects, maintaining daily and longer-term schedules, searching for and reading articles, and staying updated on multidisciplinary topics related to a study domain.

Here, I describe some of the best apps for researchers that can are available for free and can simplify both day-to-day tasks and research-related activities.

• Trello: Streamline individual and collaborative projects

Researchers need to keep track of various activities to optimize their productivity. A useful app for researchers, Trello  is a user-friendly app wherein one can create work boards for different projects and populate them with tasks. The user can assign deadlines and keep updating ongoing progress. Work boards can be shared with all team members, thereby smoothening collaborative working.

Trello offers tools to coordinate tasks among members working remotely or disparately, say, team members on the field and those in the lab. This is a particularly useful app for researchers involved in large projects, working with researchers situated across the globe. With this app you can track team projects and monitor in detail the progress of tasks under way. This can be counted among the best apps for researchers as it enables the visualization of workflows, providing team members with a comprehensive overview of a project from beginning to end. Integration with other applications allows users to access features in Gmail or other apps directly from Trello.

Key features

• Easy to use app for researchers
• Flexible and versatile
• Helps manage collaborative projects
• Evernote: Organize your thoughts and ideas

Are you still relying on Post-its and notepads? Do you jot down sudden ideas on a napkin in a coffee shop or on a scrap of paper while working in the lab? Note-taking apps like  Evernote  can help you make lists and take notes and never lose them. A handy app for researchers, Evernote helps you store all your ideas and thoughts, to-do lists, notes, and research links in one place. You can keep all these bits and bobs organized by creating separate tags and folders for different purposes.

The  Evernote web clipper  is a useful feature for saving articles, web pages, or screen grabs from the internet. You can save a page or highlights to your Evernote notebook along with any notes you want to make about that page. It allows you to sync your notes to all your devices, enabling you to organize your notes across multiple platforms. Further, this is one of the best apps for researchers because its multimedia features let you annotate images, embed files and pictures in your notes, and even record audio and video notes.

• Maintains notes, articles, and other content in one place
• Facilitates content clipping from the web
• Auto-sync across devices with this app for researchers
• R Discovery: Search Less, Read More

Researchers spend a large part of their time wading through a sea of literature, sifting out the relevant from the irrelevant.  R Discovery  is a free literature discovery app and is a great platform that lets you identify the most relevant academic research papers from top journals and publishers. This reading app for research papers covers all major disciplines in the arts and sciences.

R Discovery offers customized research reading, that is, once you set up your areas of interest, the app for research papers finds the top 3 reads and presents them in the form of a daily feed for you. Powered by AI, it learns your reading interests and finds matching relevant papers. It even provides on-the-go updates on recently published articles through notifications and email alerts making it one of the best apps for researchers. R Discovery offers a weekly roundup of the 5 latest articles and summaries of research articles from trending topics. With this intuitive app for research papers, you need not worry about a crucial article slipping through the cracks when you weren’t looking!

When you feed in key terms, the app “deep-dives” into the topics and offers articles, which you can sort by recency or relevance. It even helps you look for similar papers and bookmark important research papers. Mobile and web integration lets you read your bookmarked articles on the  R Discovery  website. What’s more, to customize your feed, you can even import your reading library from Mendeley and Zotero making this a must have app for researchers.

R Discovery can be considered a literature search and reading app for researchers everywhere that steers you in the right direction during your academic voyage!

• Curates 96+ million research articles, including over 24 million open access articles
• Intuitively provides key highlights, summaries, and roundups of research relevant to one’s field
• Integration with reference managers enables the R Discovery app for researchers to make better recommendations.

• Mendeley: Handle reference lists without getting bogged down

The thought of sorting and drawing up a reference list and formatting the in-text citations and references can make the most seasoned scholars break out into a sweat. However, useful apps for researchers like  Mendeley  take the load off these painstaking tasks and offer much more. Mendeley is a free reference management app that automatically generates bibliographies as you write. You don’t need to manually type references; Mendeley imports and organizes them in a systematic manner. The app for researchers allows you to insert citations and create reference lists in different journal formats rapidly and seamlessly. These features free up lot of time, which can be used to focus on paper or thesis writing.

Mendeley is available in both mobile and desktop formats, and researchers can conveniently read content on the go and even highlight text that they might want to return to. Researchers can export papers from the R Discovery app to Mendeley and Zotero. Once exported, they can find these papers in their libraries on ref managers. They can also connect their Mendeley and Zotero accounts with R Discovery (when they are setting up preferences on R Discovery). This will allow for this smart app for researchers to suggest topics based on the Mendeley or Zotero reading list.

• Helps annotate and organize documents
• Can be used across platforms seamlessly
• Integrates with literature discovery apps for researchers like R Discovery.
• Calm: Prevent burnout and focus on self-care

Researchers work for long hours, juggling multiple research tasks, securing funding, and dealing with stressors like dealing with harsh peer reviewer comments and article rejection. These factors can add up and affect a researcher’s mental well-being and motivation. Some  indicators can point to one’s need to focus on self-care , such as altered sleep patterns. A stressed researcher will constantly feel tired and be less efficient at work.

Calm  is a popular app for researchers looking to help reduce stress and anxiety, improve sleep quality, and aid in relaxation and self-improvement. The app provides sessions for guided meditation and breathing and masterclasses for managing stress, enhancing creativity, and much more. The use of such apps can even improve concentration and mindfulness. This is a great app for researchers to use to remain calm in the face of high work pressure, roadblocks in your experiments, and creative blocks when writing papers. A few minutes a day on such an app can help you break negative patterns. After all, a happy researcher is a productive researcher!

• Tracks a user’s basic statistics, e.g., minutes of meditation
• Sends meditation reminders
• Offers masterclasses taught by experts in the field of wellness.

Don’t worry, be “appy”!

Technological innovations like AI are constantly improving apps in terms of functions and user experience. While some apps for researchers help to ease the workload or aid in multitasking, others help in self-improvement and time management to let researchers focus better on core tasks. When possible, one should use some of these best apps for researchers to become savvier and more efficient, getting the most out of their time and effort.

R Discovery is a literature search and research reading platform that accelerates your research discovery journey by keeping you updated on the latest, most relevant scholarly content. With 250M+ research articles sourced from trusted aggregators like CrossRef, Unpaywall, PubMed, PubMed Central, Open Alex and top publishing houses like Springer Nature, JAMA, IOP, Taylor & Francis, NEJM, BMJ, Karger, SAGE, Emerald Publishing and more, R Discovery puts a world of research at your fingertips.  

Try R Discovery Prime FREE for 1 week or upgrade at just US$72 a year to access premium features that let you listen to research on the go, read in your language, collaborate with peers, auto sync with reference managers, and much more. Choose a simpler, smarter way to find and read research – Download the app and start your free 7-day trial today !  

Related Posts

What is Cluster Sampling? Definition, Method, and Examples

What is Thematic Analysis and How to Do It (with Examples)

30 Tools and Resources for Academic Research

Merriam-Webster defines “research” as “careful or diligent search; studious inquiry or examination; the collecting of information about a particular subject”. It’s not easy to conduct academic research, so here we round up 30 academic tools and resources that will facilitate your research in managing, indexing, and web scraping .

Looking for data resources for your research? Find it in  70 Amazing Free Data Resources you should know, covering government, crime, health, finances, social media, journalism, real estate, etc.

10 Research Management Tools

1. marginnote.

License: Commercial

MarginNote is a powerful reading tool for learners. Whether you are a student, a teacher, a researcher, a lawyer, or someone with a curious mind to learn, MarginNote can help you quickly organize, study and manage large volumes of PDFs and EPUBs. All-in-one learning app enables you to highlight PDF and EPUB, take notes, create the mind map, review flashcards, and saves you from switching endlessly between different Apps. It is available on Mac, iPad, and iPhone.

License: Free

Zotero is a free, easy-to-use tool to help you collect, organize, cite, and share research. It is available for Mac, Windows, and Linux. It supports managing bibliographic data and related research materials (such as PDF files). Notable features include web browser integration, online syncing, generation of in-text citations, footnotes, and bibliographies, as well as integration with the word processors Microsoft Word and LibreOffice Writer.

3. RefWorks

RefWorks is a web-based commercial reference management software package. Users’ reference databases are stored online, allowing them to be accessed and updated from any computer with an internet connection. Institutional licenses allow universities to subscribe to RefWorks on behalf of all their students, faculty, and staff. Individual licenses are also available. The software enables linking from a user’s RefWorks account to electronic editions of journals to which the institution’s library subscribes.

EndNote is the industry standard software tool for publishing and managing bibliographies, citations, and references on the Windows and Macintosh desktop. EndNote X9 is the reference management software that not only frees you from the tedious work of manually collecting and curating your research materials and formatting bibliographies, but also gives you greater ease and control in coordinating with your colleagues.

5. Mendeley

Mendeley Desktop is free academic software (Windows, Mac, Linux) for organizing and sharing research papers and generating bibliographies with 1GB of free online storage to automatically back up and synchronize your library across desktop, web, and mobile.

6. Readcube

ReadCube is a desktop and browser-based program for managing, annotating, and accessing academic research articles. It can sync your entire library including notes, lists, annotations, and even highlights across all of your devices including your desktop (Mac/PC), mobile devices (iOS/Android/Kindle), or even through the Web.

Qiqqa is a free research and reference manager. Its free version supports supercharged PDF management, annotation reports, expedition, Ad-supported, and 2GB free online storage.

Docear offers a single-section user interface that allows the most comprehensive organization of your literature; a literature suite concept that combines several tools in a single application (pdf management, reference management, mind mapping, …); A recommender system that helps you to discover new literature: Docear recommends papers which are free, in full-text, instantly to download, and tailored to your information needs.

9. Paperpile

Paperpile is a web-based commercial reference management software, with a special emphasis on integration with Google Docs and Google Scholar. Parts of Paperpile are implemented as a Google Chrome browser extension

JabRef is an open-source bibliography reference manager. The native file format used by JabRef is BibTeX, the standard LaTeX bibliography format. JabRef is a desktop application that runs on the Java VM (version 8), and works equally well on Windows, Linux, and Mac OS X. Entries can be searched in external databases and BibTeX entries can be fetched from there. Example sources include arXiv, CiteseerX, Google Scholar, Medline, GVK, IEEEXplore, and Springer.

10 Reference/Index Resources

1. google scholar.

Google Scholar is a freely accessible web search engine that indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines. It includes most peer-reviewed online academic journals and books, conference papers, theses and dissertations, preprints, abstracts, technical reports, and other scholarly literature, including court opinions and patents. You can extract these academic data easily by scraping Google Scholar search results .

arXiv (pronounced “archive”) is a repository of electronic preprints (known as e-prints) approved for publication after moderation, that consists of scientific papers in the fields of mathematics, physics, astronomy, electrical engineering, computer science, quantitative biology, statistics, and quantitative finance, which can be accessed online. In many fields of mathematics and physics, almost all scientific papers are self-archived on the arXiv repository.

3. Springer

Springer Science+Business Media or Springer, part of Springer Nature, has published more than 2,900 journals and 290,000 books, which covers science, humanities, technical and medical, etc.

4. Hyper Articles en Ligne

Hyper Articles en Ligne (HAL) is an open archive where authors can deposit scholarly documents from all academic fields, run by the Centre pour la Communication Scientifique direct, which is part of the French National Centre for Scientific Research. An uploaded document does not need to have been published or even to be intended for publication. It may be posted to HAL as long as its scientific content justifies it.

MEDLINE (Medical Literature Analysis and Retrieval System Online, or MEDLARS Online) is a bibliographic database of life sciences and biomedical information. It includes bibliographic information for articles from academic journals covering medicine, nursing, pharmacy, dentistry, veterinary medicine, and health care. MEDLINE also covers much of the literature in biology and biochemistry, as well as fields such as molecular evolution.

Compiled by the United States National Library of Medicine (NLM), MEDLINE is freely available on the Internet and searchable via PubMed and NLM’s National Center for Biotechnology Information’s Entrez system.

6. ResearchGate

ResearchGate is a social networking site for scientists and researchers[3] to share papers, ask and answer questions, and find collaborators.[4] According to a study by Nature and an article in Times Higher Education, it is the largest academic social network in terms of active users.

7. CiteSeerx

Owner: Pennsylvania State University

CiteSeerx ( CiteSeer ) is a public search engine and digital library for scientific and academic papers, primarily in the fields of computer and information science. Many consider it to be the first academic paper search engine and the first automated citation indexing system. CiteSeer holds a United States patent # 6289342, titled “Autonomous citation indexing and literature browsing using citation context”.

Owner: Elsevier

Scopus is the world’s largest abstract and citation database of peer-reviewed research literature. With over 22,000 titles from more than 5,000 international publishers. You can use this free author lookup to search for any author; or, use the Author Feedback Wizard to verify your Scopus Author Profile.

9. Emerald Group Publishing

Emerald Publishing was founded in 1967, and now manages a portfolio of nearly 300 journals, more than 2,500 books, and over 1,500 teaching cases, covering the fields of management, business, education, library studies, health care, and engineering.

10. Web of Science

Owner: Clarivate Analytics (United States)

Web of Science (previously known as Web of Knowledge) is an online subscription-based scientific citation indexing service originally produced by the Institute for Scientific Information (ISI)

10 Information Collection Tools

This part divides the 10 information collection tools into 5 information survey tools and 5 data collection tools for further use.

5 Information Survey Tools

1. google forms.

Google Forms is a simple option for you if you already have a Google account. It supports menu search, a shuffle of questions for randomized order, limiting responses to once per person, custom themes, automatically generating answer suggestions when creating forms, and an “Upload file” option for users answering to share content through.

Moreover, the response can be synced in Google Drive, and users can request file uploads from individuals outside their respective companies, with the storage cap initially set at 1 GB.

2. Survey Monkey

Survey Monkey is quite a well-known name in the field but is also costing. It is a great choice for you if you want an easy user interface for basic surveys, as its free plan supports unlimited surveys, however, each survey is limited to 10 questions.

3. Survey Gizmo

SurveyGizmo can be customized to meet a wide range of data-collection demands. The free version has up to 25 question types, letting you write a survey that caters to specific needs. It also offers nearly 100 different question types that can all be customized to the user’s liking.

4. PollDaddy

PollDaddy is online survey software that allows users to embed surveys on their website or invite respondents via email. Its free version supports unlimited polls, 19 types of questions, and even adding images, videos, and content from YouTube, Flickr, Google Maps, and more.

5. LimeSurvey

LimeSurvey is an open-source survey software as a professional SaaS solution or as a self-hosted Community Edition. LimeSurvey’s professional free version provides 25 responses/month with an unlimited number of surveys, unlimited administrators, and 10 MB of upload storage.

5 Web Data Collection Tools

1. octoparse.

Octoparse is the most easy-to-use web scraping tool for people without a prior tech background. It is widely used among online sellers, marketers, researchers, and data analysts. With its intuitive interface, you can scrape web data within points and clicks. If you are looking for a one-stop data solution, Octoparse also provides a  web data service . Or you can simply follow the Octoparse user guide to scrape website data easily for free. It also provides ready-to-use web scraping templates to extract data from Amazon, eBay, Twitter, BestBuy, etc.

https://www.octoparse.com/template/email-social-media-scraper

Its free version offers unlimited pages per crawl, 10 crawlers, and up to 10,000 records per export. If the data collected is over 10,000, then you can pay $5.9 to export all the data. If you need to track the dynamic data in real time, you may want to use Octoparse’s premium feature: scheduled cloud extraction. Read its customer stories to get an idea of how web scraping enhances businesses.

2. Parsehub

Parsehub is another non-programmer-friendly desktop software for web scraping, which is available to various systems such as Windows, Mac OS X, and Linux. Its free version offers 200 pages per crawl, 5 public projects, and 14 days for data retention.

3. Docparser

Docparser converts PDF documents into structured and easy-to-handle data, which allows you to extract specific data fields from PDFs and scanned documents, convert PDF to text, PDF to JSON, PDF to XML, convert PDF tables into CSV or Excel, etc. Its starting price is $19, which includes 100 parsing credits.

Scrapy is an open-source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.

Feedity automagically extracts relevant content & data from public web pages to create auto-updating RSS feeds. Instantly convert online news, articles, discussion forums, reviews, jobs, events, products, blogs, press releases, social media posts, or any other Web content into subscribable or publishable notifications. The starter version offers 20 feeds and 6 hours update interval, with a cost of $9 per month.

Final Thoughts

Hope it’s helpful for you after learning the top 30 tools for academic research. Data is becoming more and more important in today’s world, not only for academic research but also for other industries. The basic step of data analysis is data collection, so using a web scraping tool like Octoparse can really save you time and energy.

Explore topics

• # Web Scraping 187
• # Knowledge 78
• # Data Collection 40
• # Octoparse 34
• # Big Data 31
• # Customer Reviews 0

Get started with Octoparse today

Related articles.

9 Must-Have Online Tools for Researchers

Your changes have been saved

Email Is sent

Please verify your email address.

You’ve reached your account maximum for followed topics.

Research is already time-intensive work. And little tasks like formatting or citing eat up more of your time. Luckily, you can automate these tedious tasks to a large extent and focus more on actual research.

Here, we’ve rounded up all the online tools researchers should have in their arsenal—from Google Scholar to Citationsy.

1. Google Scholar

Google Scholar is perhaps the most popular tool for finding scholarly literature on a plethora of topics. The search engine makes it simple for anyone to explore academic papers, theses, case law, books, etc.

On the search results page, you can view the author name, journal title, and total citations, which can help you gauge the credibility of the paper. Similarly, you can view related articles to explore the topic in detail.

Besides helping you find relevant information, Google Scholar lets you organize articles in your library. Create labels, sort by dates, and maintain a reading list to keep up with the latest research in your field.

You can also create your Google Scholar profile to show your work on Google Scholar and track citations to your work. Although it is an excellent tool, there are alternatives to Google Scholar in case it doesn’t work for you.

Zotero describes itself as “your personal research assistant”, and we tend to agree. The tool cuts down on several monotonous tasks, like collecting research sources and adding citations.

If you have saved hundreds of information sources, finding the information you need can be difficult. Zotero solves this problem by letting you organize files into categories and assign keywords. With each saved item, you can add notes, attachments, and related material.

Zotero also simplifies the referencing by creating citations and bibliographies. Although the Zotero desktop client offers more features, the browser extension can save sources in the online library, letting you organize, tag, and cite them.

Best of all, it allows collaboration and sharing of documents. Zotero is a free, open-source project. However, you'll need to pay for storage if the 300 MB free plan doesn’t work for you.

Download: Zotero for Google Chrome | Microsoft Edge (Free, Paid)

3. ResearchGate

Although most researchers are already familiar with ResearchGate, the platform deserves a mention here. It’s a networking platform for researchers, having over 20 million accounts.

You can follow other researchers, share your work, and ask questions from domain experts. Similarly, you can showcase your research and projects to a wider audience and receive feedback. Based on your profile, ResearchGate can also help you connect with potential job opportunities.

With over 135 million pages of scholarly literature, ResearchGate helps you explore publications on various topics and follow specific projects/questions. Creating an account here is free. But, you’ll need to have an institutional email address or go through additional checks to ensure you’re a researcher.

4. Mendeley

Mendeley is much similar to Zotero, as it allows you to save papers, organize your library, and add citations in various styles.

What sets it apart though is that it has a user-friendly interface and a search engine for finding research papers. You can narrow the results down by year, journal title, author, and document type.

Directly from the search results page, you can add papers to your collection and view Open Access papers. Likewise, Mendeley Data lets you find research datasets.

Mendeley has a career section with thousands of listed vacancies—a great resource for finding technical jobs. It is available, both as a desktop application and a web-based tool (with browser extension). The free plan allows you to store 2 GB of data, after which you'll have to pay $4.99 per month or more, depending on your needs.

Download: Mendeley for Google Chrome | Firefox (Free, Paid)

5. SciSpace

If you spend hours on getting the formatting correct, SciSpace is for you. This awesome web-based editor makes it simple to write and format your research papers, thanks to the huge collection of templates.

From a single editor screen, you can write, format, add citations, check plagiarism, and insert tables. You can collaborate with other researchers directly using SciSpace.

Besides publishing tools, SciSpace can help you discover scholarly literature and has an index of over 270 million papers. So you can search for papers by topic, authors, journals, and institutions.

SciSpace has a free plan, albeit with quite limited features. So if you’re serious, you can subscribe to SciSpace for $20 per month.

6. Turnitin

Before submitting your research work, it’s necessary to run a plagiarism report and ensure the content is original.

Unless you've specific requirements about the software, Turnitin works great. It can detect plagiarism in academic work, thanks to its gigantic library of published papers.

The iThenticate service is specially designed for publishers. Using this tool, you can make adjustments to similarity criteria and view attached sources. For researchers who work in a group, it allows sharing folders.

Turnitin is a paid tool and only has an institutional license. So, you'll have to request quotes.

8. Hypothesis

Hypothesis is a Chrome extension that allows you to highlight documents and add notes. However, the main feature of this tool is the ability to collaborate seamlessly with your coworkers. You can add people to a group, share the document with them, and annotate documents.

Hypothesis is free to use. Although it looks like a simple extension, it’s one of the best tools to annotate web pages for research .

9. Citationsy

If you hate adding citations, try using Citationsy. The intuitive reference manager can add citations and references in various styles automatically. You've to only select the journal and article in the Citations section.

Whether you want to cite a journal article, book, website, or podcast, Citationsy makes it easy. You can share the bibliographies with your fellow researchers.

Citationsy costs $9.99 per month and $4.99 per month for students. It has browser extensions as well as mobile applications.

Simplify Your Research Workflow

We get it. Research ain’t an easy task. But perhaps what is more frustrating are all the little tasks that add little value. But thanks to these tools, you boost your productivity, collaborate seamlessly with your co-workers, and focus on the actual research.

In case you’ve just stepped into the field of research, there are tools to help you polish your skills as well.

• Online Tools
• Website Lists

Search form

Department of history.

Make a Gift

• Robert D. Cross Memorial Lecture Series
• Faculty Resources
• Emeritus Faculty
• Affiliated Faculty
• Graduate Students
• Department Staff
• Department Contacts
• Office Hours
• Guide to Graduate Study in History
• Fields of Study for the General Exam
• Prospective Graduate Students
• Dissertations in Progress
• Placement History
• BA/MA 4+1 Program
• Distinguished Majors Program
• Research Opportunities & Awards
• Transfer Credit
• History Teachers Education
• Graduation Details
• Concentrations
• Publications
• The Caribbean
• Europe & Russia
• Global & International
• Latin America
• The Middle East
• US & Canada
• Ancient & Classical (pre-500)
• Medieval & Post-Classical (c.500-1400)
• Early Modern (c.1400-1800)
• 19th Century
• 20th Century
• Cultural & Social History
• Digital Humanities
• Economic History & History of Capitalism
• Empires & Colonialism
• Environmental History
• Genocide & Violence
• History of Slavery
• Intellectual History & History of Ideas
• Jewish History
• Labor History
• Legal History
• Material Culture
• Military & War History
• Political History
• Race & Ethnicity
• Religious History
• Science, Medicine, & Technology
• Spatial History, Frontiers, & Migration
• Transnational & Diplomatic History
• Women, Gender, & Sexuality
• History Course Concentrations per Semester
• Spring 2024
• Spring 2023
• Recently Offered Courses
• Full Course Catalog

Software for Research and Digital Notetaking

Most of us utilize a range of media when conducting research—we gather documents from online databases, as well as images, audio, and video files from the web. We review box after box of physical documents at the archives. We take notes from books as well as from journal articles we’ve downloaded as pdfs. Some of us conduct oral history interviews, by person or phone. It can be a challenge to keep so much diverse material organized, accessible, and easy to work with when you’re in the process of writing a major research paper or dissertation.

Fortunately, there are a wide array of digital tools for notetaking, generating citations, and organizing your research. Listed below are a few of the many available options:

A free, easy-to-use tool to help you collect, organize, cite, and share your research sources from your browser. Integrates with Word to generate bibliographies and citations quickly.

PhD dissertation writing software. Helps to retain, organize, and evaluate assertions, ideas, and concepts for your dissertation, prepares outline-structured notes and saves your time by eliminating tedious cut-and-paste work.

Scrivener is a powerful content-generation tool for writers that allows you to concentrate on composing and structuring long and difficult documents. Scrivener puts everything you need for structuring, writing and editing long documents at your fingertips. On the left of the window, the “binder” allows you to navigate between the different parts of your manuscript, your notes, and research materials, with ease. No more switching between multiple applications to refer to research files: keep all of your background material—images, PDF files, movies, web pages, sound files—right inside Scrivener.

Mendeley is a free reference manager and academic social network that can help you organize your research, collaborate with others online, and discover the latest research. The program automatically generate bibliographies, imports papers from other research software, helps you find papers based on what you are reading, and enables you to collaborate with other researchers online.

FileMakerPro

Powerful database software for organizing your research (available for Mac and PC). Take notes, import pdfs, images, videos, and audio files, cross-reference your documents, and more. FilemakerPro comes with built in templates for research notes, but it is completely customizable as well. Fully-searchable. Take advantage of a significant student discount by purchasing this software through the UVa bookstore.

Intuitive notetaking software that comes with Microsoft Office. OneNote is a digital notebook that provides a single place where you can gather all of your notes and information, with the added benefits of powerful search capabilities to find what you are looking for quickly, plus easy-to-use shared notebooks so you can manage information overload and work together with others more effectively. You can insert almost anything into a page, and create as many pages as you want in a notebook. Use to take notes and organize documents, images, websites and more in a user-friendly program with a great visual interface. (For PCs)

GrowlyBird Notes

The closest approximation to OneNote for the Mac--and it's free.Growly Notes lets you capture everything you’re interested in, all in one place. Pages can contain almost anything: formatted text, images, movies, audio clips, PDF files, tables, lists, web and file links, and drawings you create in Notes. There are no rules for where things have to go: put an image beside text or under it. Draw shapes on top of other notes. Put two snippets of text right next to each other. Click anywhere and start typing.

Omeka is a free, flexible, and open source web-publishing platform for the display of library, museum, archives, and scholarly collections and exhibitions. Its “five-minute setup” makes launching an online exhibition as easy as launching a blog.

A.nnotate is an online annotation, collaboration and indexing system for documents and images, supporting PDF and MS Office formats.

A reference manager for Mac and iOS users. Bookends can perform Internet searches to retrieve references and associated pdfs or web pages, or immediately find and import references for which you already have the pdf.

Unfortunately we don't fully support your browser. If you have the option to, please upgrade to a newer version or use Mozilla Firefox , Microsoft Edge , Google Chrome , or Safari 14 or newer. If you are unable to, and need support, please send us your feedback .

We'd appreciate your feedback. Tell us what you think! opens in new tab/window

Sharing research data

As a researcher, you are increasingly encouraged, or even mandated, to make your research data available, accessible, discoverable and usable.

Sharing research data is something we are passionate about too, so we’ve created this short video and written guide to help you get started.

Research Data

What is research data.

While the definition often differs per field, generally, research data refers to the results of observations or experiments that validate your research findings. These span a range of useful materials associated with your research project, including:

Raw or processed data files

Research data  does not  include text in manuscript or final published article form, or data or other materials submitted and published as part of a journal article.

Why should I share my research data?

There are so many good reasons. We’ve listed just a few:

How you benefit

You get credit for the work you've done

Leads to more citations! 1

Can boost your number of publications

Increases your exposure and may lead to new collaborations

What it means for the research community

It's easy to reuse and reinterpret your data

Duplication of experiments can be avoided

New insights can be gained, sparking new lines of inquiry

Empowers replication

And society at large…

Greater transparency boosts public faith in research

Can play a role in guiding government policy

Improves access to research for those outside health and academia

Benefits the public purse as funding of repeat work is reduced

How do I share my research data?

The good news is it’s easy.

Yet to submit your research article?  There are a number of options available. These may vary depending on the journal you have chosen, so be sure to read the  Research Data  section in its  Guide for Authors  before you begin.

Already published your research article?  No problem – it’s never too late to share the research data associated with it.

Two of the most popular data sharing routes are:

Publishing a research elements article

These brief, peer-reviewed articles complement full research papers and are an easy way to receive proper credit and recognition for the work you have done. Research elements are research outputs that have come about as a result of following the research cycle – this includes things like data, methods and protocols, software, hardware and more.

You can publish research elements articles in several different Elsevier journals, including  our suite of dedicated Research Elements journals . They are easy to submit, are subject to a peer review process, receive a DOI and are fully citable. They also make your work more sharable, discoverable, comprehensible, reusable and reproducible.

The accompanying raw data can still be placed in a repository of your choice (see below).

Uploading your data to a repository like Mendeley Data

Mendeley Data is a certified, free-to-use repository that hosts open data from all disciplines, whatever its format (e.g. raw and processed data, tables, codes and software). With many Elsevier journals, it’s possible to upload and store your data to Mendeley Data during the manuscript submission process. You can also upload your data directly to the repository. In each case, your data will receive a DOI, making it independently citable and it can be linked to any associated article on ScienceDirect, making it easy for readers to find and reuse.

View an article featuring Mendeley data opens in new tab/window  (just select the  Research Data  link in the left-hand bar or scroll down the page).

What if I can’t submit my research data?

Data statements offer transparency.

We understand that there are times when the data is simply not available to post or there are good reasons why it shouldn’t be shared.  A number of Elsevier journals encourage authors to submit a data statement alongside their manuscript. This statement allows you to clearly explain the data you’ve used in the article and the reasons why it might not be available.  The statement will appear with the article on ScienceDirect. 

View a sample data statement opens in new tab/window  (just select the  Research Data  link in the left-hand bar or scroll down the page).

Showcasing your research data on ScienceDirect

We have 3 top tips to help you maximize the impact of your data in your article on ScienceDirect.

Link with data repositories

You can create bidirectional links between any data repositories you’ve used to store your data and your online article. If you’ve published a data article, you can link to that too.

Enrich with interactive data visualizations

The days of being confined to static visuals are over. Our in-article interactive viewers let readers delve into the data with helpful functions such as zoom, configurable display options and full screen mode.

Cite your research data

Get credit for your work by citing your research data in your article and adding a data reference to the reference list. This ensures you are recognized for the data you shared and/or used in your research. Read the  References  section in your chosen journal’s  Guide for Authors  for more information.

Ready to get started?

If you have yet to publish your research paper, the first step is to find the right journal for your submission and read the  Guide for Authors .

Find a journal by matching paper title and abstract of your manuscript in Elsevier's  JournalFinder opens in new tab/window

Find journal by title opens in new tab/window

Already published? Just view the options for sharing your research data above.

1 Several studies have now shown that making data available for an article increases article citations.

• Best Agile Project Management Software

Best Agile Project Management Software Of 2024

Updated: Jul 22, 2024, 8:14am

An Agile project management solution should support the most popular methodologies: Kanban and Scrum. We considered a variety of tools that enable teams to track tasks and backlogs, manage projects from various views and adopt the software quickly and easily. To determine which is the best Agile project management software, we looked at the value-to-cost ratio, ease of use and flexibility, so it isn’t just your Scrum teams that are able to use it.

• Best Construction Project Management Software
• Best Project Portfolio Management Software
• Best Gantt Chart Software
• Best Task Management Software

Featured Partners

From $8 monthly per user

Zoom, LinkedIn, Adobe, Salesforce and more

On monday.com’s Website

Yes, for one user and two editors

$9 per user per month

Google Drive, Slack, Tableau, Miro, Zapier and more

On Smartsheet’s Website

Yes, for unlimited members

$7 per month

Slack, Microsoft Outlook, HubSpot, Salesforce, Timely, Google Drive and more

On ClickUp’s Website

$9.80 per user per month

Salesforce, Adobe, Miro, Netsuite, Quickbooks, SAP

On Wrike’s Website

Forbes Advisor Ratings

How to choose the best agile project management software, methodology, frequently asked questions (faqs).

• ClickUp : Best for Flexibility
• Teamwork.com : Best for Agencies
• Asana : Best for Unlimited Features
• monday.com : Best for Ease of Use
• Airtable : Best for Customization
• Jira : Best for Bug Tracking
• Smartsheet : Best for Templates
• Trello : Best for Beginners
• Wrike : Best for Multiple Departments

Why You Can Trust Forbes Advisor Small Business

The Forbes Advisor Small Business team is committed to bringing you unbiased rankings and information with full editorial independence. We use product data, first-person testing, strategic methodologies and expert insights to inform all of our content to guide you in making the best decisions for your business journey.

• 30 providers ranked
• 64 metrics applied across eight weighted categories
• 1,920 data points collected

Best for Flexibility

Starting Price

$7 per user per month

(billed annually)

Kanban Boards

Unique Features

Workload management, video recording and whiteboards

ClickUp is a flexible project management solution with features that work for any methodology you choose, but its Agile project management features stand out compared to its competitors. From ideation to iterations, ClickUp gives you tools to help you go from whiteboard plans to created tasks. The workload management tool gives you a clear view of which resources you have available.

Most of the best project management software today include a variety of project views, including Kanban, but that’s just one part of the Agile project management philosophy that provides a visual representation of tasks within a project. ClickUp can be used for Kanban, Scrum or Scrumban, as needed.

Learn more : Read our full ClickUp review .

Who should use it :

Almost any type of business can easily adopt ClickUp for any project management methodology, but its specific tools are excellent for Agile project management.

• Free plan includes sprint management
• Built-in video recording
• Collaborative whiteboards
• Native real-time chat
• Low storage allowance on free plan
• Steep learning curve

Best for Agencies

Teamwork.com.

$5.99 per user per month

Built-in team chat, time tracking and invoices

Teamwork is a versatile project management app that includes features that are ideal for agencies that work closely with clients. It also lets you choose the view you prefer so you can create workflows that work best for you. The free plan supports up to five users, so it’s a good pick for a small team that needs the basics for an agency. You can set up Teamwork for Agile project management with Kanban boards and run sprints, but the main reason to use it is for its agency features.

Time tracking is included on all plans, as are billing and invoicing features. There are also automations included for all plans, but actions are limited per month. If you opt for the entry-level plan, you can create user rates to show clients and you’ll get more automation actions, collaborative documents and a portfolio view.

Learn more : Read our full Teamwork review .

Teamwork is best for agencies that need client-based features, such as time tracking, billing and invoicing.

• Free plan offered for up to five users
• Time tracking on all plans
• Includes billing and invoicing features
• Requires three users minimum for entry-level plan
• Few reporting options in lower plans

Best for Unlimited Features

$10.99 per user per month

Lots of unlimited options and logic-based forms

One of the reasons why Asana gets high marks as an Agile project management app is that it doesn’t apply many limits to its free plan. You can create as many tasks, projects and due dates as needed. There are also no limits to messages or comments either. Storage is unlimited except per file, which is limited to 100MB. High-tier plan users can take advantage of advanced features, such as logic-branching forms, which allow you to create follow-up questions in forms. Plus you get more reporting options, which makes it much easier to manage projects based on goals, time spent and workload capacity.

Learn more : Read our full Asana review .

If you don’t want to be limited by the number of tasks, projects or comments, Asana is a good choice, even if you stick to the free plan.

• Generous free plan with few limits
• User-friendly interface
• Forms with conditional logic included
• Gantt charts only on paid plans
• Goals, time tracking reporting only on high-tier plans

Best for Ease of Use

Workflow templates and color-coded labels

We ranked monday.com high because it’s one of the easiest project management apps to use. It prioritizes visual boards and labels, which makes it easy to manage tasks, teams and projects. There are plenty of workflow templates that give you a jump start to creating Scrum boards and sprints. The color-coded labels make it simple to spot overdue tasks, track bugs and manage team members. The biggest downside to monday.com is that it requires at least three users for the entry-level paid plan, so it’s a bit more expensive than its competitors.

Learn more : Read our full monday.com review .

The monday.com platform makes the most sense for medium to large teams because of its pricing structure, and it works well for newer users who may not be familiar with Scrum and Agile methodologies.

• Free plan for up to two users
• More than 200 workflow templates
• Color-coded labels and charts
• Requires three users minimum on low-tier plan
• No integrations on free or entry-level plan

Best for Customization

$20 per user per month

Native collaboration tools and branded forms

Airtable is all about customization and flexibility. You can build your interface the way you want it, add extensions to help you create charts, tables or integrate with other apps you already use. There is a free plan that allows unlimited bases, but you can only have up to 1,000 records on each base, and storage is limited to 2GB per base.

Learn more : Read our full Airtable review .

Airtable is a very flexible project management solution for any type of business, but it is expensive, so it may not be for startups or small businesses.

• Free plan available
• Customizable, user-friendly interface
• Extensions for added functionality
• Pricey compared to industry standard
• Slight learning curve

Best for Bug Tracking

$7.08 per user per month

Strong security features and sandbox

Jira is meant for developers and it shows with its bug-tracking tools, strong security features and sandbox tool. A small team can easily use Jira to run sprints, track bugs and set task dependencies for epics and stories. There is no native Gantt chart, and collaboration is a bit difficult, but that’s because Jira is supposed to be used in conjunction with its sister program Confluence. Still, it’s a great Agile project management software for managing backlogs, customizing workflows and running Scrum for any size team.

Learn more : Read our full Jira review .

Jira was built with software developers in mind, and it includes excellent bug tracking tools.

• Free plan for up to 10 users
• Advanced task dependency on high-tier plans
• Scrum boards are standard
• No native Gantt chart

Best for Templates

Proofing, DocuSign integration and conditional logic forms

If you’re most familiar with spreadsheets for project management and you’re new to Agile as a philosophy, Smartsheet’s interface with its many templates could be a good gateway for you. You have more than 500 templates from which to choose to get started with any type of project, road map or budget, for example. Also, it supports eight languages.

A few unique features of Smartsheet include proofing and conditional logic forms, but these are both only available on the Pro plan or higher. The DocuSign integration is not common for project management software either, but it’s only on the Enterprise plan. All this is to say that Smartsheet can work well for small teams, but you’ll pay a high price for a more complete solution. Plus, the free plan limits you to two sheets, which may not work well for more than a freelancer.

Learn more : Read our full Smartsheet review .

Smartsheet makes the transition to Agile project management a bit easier with pre-built templates, so it’s a good choice for newcomers who are more comfortable with spreadsheets.

• Customizable templates
• Automation on all plans
• Very limited free plan
• Pricey add-ons

Best for Beginners

$5 per user per month

Power-Ups and automations

Trello is all about Kanban boards, so it’s a good option for Agile project management. It also has one of the most user-friendly interfaces that anyone can pick up quickly. In addition to being easy to use, Trello is generous with its free plan and affordable with its paid plans. Although it is a simple solution, there are Power-Ups, which are Trello-branded extensions that can add automation or new features to your boards.

Learn more : Read our full Trello review .

Trello is a user-friendly Agile project management software that is affordable, so it’s great for teams that are new to Kanban and Scrum methodologies.

• Free for unlimited users
• Power-Ups extend functionality
• Intuitive interface
• Not as advanced as industry standard
• Hard to track multiple projects

Best for Multiple Departments

Folder hierarchy, AI-assisted features and custom forms

Wrike is an all-in-one project management solution that is highly customizable, so it works well for any department. Its unique features include machine learning tools that create tasks and subtasks automatically based on your former tasks, which are included on all plans. There’s also artificial intelligence-assisted project prediction, which helps you plan for any potential failures or blocks. There are plenty of customization options, but some are available only in higher-tier plans, such as custom request forms and custom approval flows.

Learn more : Read our full Wrike review .

Wrike can be configured for any type of department, regardless of whether you prefer Agile project management or Waterfall, so it’s a good choice for a large company with varied needs.

• Free plan for unlimited users
• Machine learning and AI features
• Collaborators allowed on all plans
• Expensive compared to industry standard
• No Gantt chart on free plan

Some of the most important features for Agile project management can be found in all-in-one project management apps. Consider how your team works and what it needs before choosing a specific Agile project management app or something more flexible that can work for multiple departments.

Essential Features

• Kanban Board : Cards devoted to tasks that can be moved from column to column keep sprints organized and show progress at a glance. This is helpful for managing backlogs, tasks and running sprints.
• Task Management : This is a feature you should find in any project management system. Without it, you won’t be able to track the progress of work or who’s doing what.
• In-App Communication : Whether it’s chat, comments or audio, it’s important to have some form of communication in the platform to reduce back-and-forth communication via email or meetings.
• Reporting : Most project management software include pre-built or custom reports, so you can track a variety of data in relation to your projects. This can help you head off issues at the pass or see where some people may be at capacity, for example.

Ease of Use

Depending on your team’s experience with project management software, it’s best to choose a simple platform. The easier it is for new users to adopt software, the more likely they are to continue using it.

Flexibility

A flexible platform makes it easy for any department to make use of the same software as a team that requires an Agile approach. Look for all-in-one options that allow customization and automation for the best fit for all users.

On monday.com's Website

On Smartsheet's Website

On ClickUp's Website

On Wrike's Website

At Forbes Advisor, we do everything possible to bring you the fairest reviews of software. In fact, we have specific methodologies in place to ensure we perform the research, hands-on testing and deep dives into user reviews to ascribe a one through five ranking to each platform.

To determine the best Agile project management software, we considered the same factors as in our best project management software, but we narrowed the scope a bit. If a provider doesn’t offer Kanban or Scrum boards, we nix it from the list. We also considered flexibility, Gantt charts and overall ease of use. This is the breakdown of how we scored each title:

• Pricing and Fees (8%) : Pricing is always going to be a huge factor for small businesses, which is why we give it 8% of the total score.
• Project Management Features (28%) : Project management features include a variety of factors including the ability to assign roles to users, utilize various views (Kanban, Gantt, etc.) and create roadmaps. We assigned 28% of the ranking to this section because it’s what makes Agile project management what it is.
• Organizing Features (15%) : Organization features make up 15% of the score and include extras, such as subtasks, milestones, multiple assignees and task IDs.
• Technical Features (19%) : We evaluated providers based on their technical features, such as the availability of a mobile app, task automation, advanced search function and more.
• Collaboration Features (7%) : We reviewed each provider for its ability to generate proofs and invoices, integrate with a variety of third-party applications and work collaboratively on documents, to name just a few factors from this category.
• Ratings and Reviews (5%) : We considered what users have to say about each platform, both positive and negative reviews, on reputable user review sites.
• Service and Support (8%) : We took into account the support you get from each software, whether it’s by phone, live chat or only help centers.
• Expert Score (10%) : To determine the expert score, we focused on the less tangible features of each platform; the qualities you can only judge by using each app. Ten percent of the score is dedicated to ease of use, value, popularity and standout features.

What is the most popular tool used in Agile?

Without question, it is an Agile project management app that empowers teams to run sprints and manage backlogs. Our pick for the best Agile project management tool is ClickUp, but Teamwork, Asana and monday.com are also excellent choices.

What is the difference between Agile and Scrum?

The difference between Agile and Scrum is that Agile is a project management philosophy while Scrum is an Agile methodology. To go a step further, consider Kanban vs. Scrum : Kanban is a simplified Agile methodology and differs from Scrum in that it doesn’t require sprint planning.

Is Jira Agile or Scrum?

It’s both. You can use Jira to run Agile methodologies such as Scrum or Kanban. The biggest difference between a Scrum board and a Kanban board is that a Scrum board includes a backlog, daily stand-ups, planning and retrospectives, while a Kanban board tends to include what needs to be done, what’s being done and what’s done. Both are Agile frameworks that work well for continuous work.

Next Up In Project Management

• Best Project Management Software
• Best Asana Competitors & Alternatives
• Best Scheduling Apps
• monday.com Review
• Asana Review
• Trello Review

Amy Nichol Smith spent more than 20 years working as a journalist for TV and newspapers before transitioning to software and hardware product reviews for consumers and small businesses. She has been featured in publications such as L.A. Times, Tom's Guide, Investopedia and various newspapers across the U.S.

The Best Review Management Software to Boost Your Business in 2024

Published: July 24, 2024

Getting reviews is one of the most important aspects of growing your business—they’re a barometer of how you’re doing and are invaluable as a marketing tool for many reasons. Yet, as a marketing strategist, so many of my clients haven’t baked getting reviews into their process and struggle to follow up consistently. That’s where review management software comes in.

Most of my projects involve working with clients to get reviews so we have effective voice of customer data and testimonials to build trust and social proof. However, it can be like pushing a boulder uphill to get reviews in a timely fashion, so to make their next project smoother sailing, I always do my best to help them integrate the process of asking for reviews at the conclusion of every project or sale.

Some automate it through their email or SMS marketing software. Others have CRMs with review management features. Still others use standalone review management software to help them track customer sentiment, improve products and customer service, and grow their sales.

Because sorting through all the options can be challenging, I’m breaking down some of the best review management software options to simplify your search.

Table of Contents

Best Review Management Software

What is review management software, benefits of review management software, how to choose customer review management software, ready, set, manage your reviews.

Review management refers to the process of getting, analyzing, and responding to customer reviews . The more complex your business (and the more platforms your customers are on), the more important it becomes to use review management software to keep track of what people are saying about you online.

For example, when I’m mining customer reviews for data, I might look at reviews on sites that include, but aren’t limited to:

• TrustPilot.

What’s more, I might also set a Google Alert for mentions of my client and their company and check out forums like Reddit.

As your company grows, keeping track of all the reviews, responding to them appropriately, and integrating the feedback with sales, marketing, and operations can become a full-time job — unless, of course, you bring in a tool to help you streamline it.

Looking for prompts or ideas for responding to reviews? Grab our FREE Review Response Templates here !

Free Review Response Templates

20 prompts to help you respond to customer complaints and comments.

• Positive Reviews
• Negative Reviews
• Mixed Reviews
• False/Slanderous Reviews

Download Free

All fields are required.

You're all set!

Click this link to access this resource at any time.

Most people today — 93% of us — are influenced by reviews when it comes to making purchasing decisions. So, while you already know reviews are important, you may not understand the benefits or, more importantly, the role review management software can play.

Makes It Easier to Get Reviews

In my opinion, the number one benefit of customer review management software is that it makes it easier to get reviews in the first place. If this is not baked into your process, you may not have enough data to make decisions. And as I’ve mentioned, having reviews is helpful on so many levels. Whatever your industry, your buyers are using reviews and personal recommendations to make decisions:

• 94% of B2B purchasers use online reviews to make decisions. ( Clutch )
• 82% of review site users say reviews are more influential than sales claims. ( B2B SaaS Reviews )
• 84% of consumers place as much trust in online reviews as in personal recommendations. ( CapitalOne Shopping Research )

And if those stats aren’t enough to help you understand the importance of reputation and review management software, here are a few more:

• 94% of businesses using reputation management tools see an ROI. ( BrightLocal )
• 75% of customers recommend companies based on a great experience. ( SalesForce Research )
• Conversion rates increase 270% when online retailers display reviews and peak with a 4.9 out of 5-star rating. ( CapitalOne Shopping Research )

So now that you understand the importance of reviews in your sales process, let’s explore some of the other benefits.

Helps Your Marketing Strategy

I can talk about marketing all day long, but let’s keep this brief. Here are some of the top marketing benefits of using review management software:

• Voice of the customer data. When you use the words your customers use to talk about their problems, it’s easier to build trust with them because you can show you understand their needs.
• Social proof. People like what other people like, so having reviews that show how incredible your products or services may tip the scale in your favor.
• Buzz building. When you get new reviews in places like Google My Business, it helps the search engines build trust in your business, plus it gives you content to share online.

But it’s also about authenticity. Anytime you can share the review in its “natural” environment, you can combat some of the fears of fake reviews. 79% of people think that fake reviews are a problem, so when you can alleviate their concerns, you’re ahead of the game.

And because different types of review management software make it easier to gather, analyze, disseminate, and respond to feedback, it can save you considerable time.

Improves the Customer Experience

In addition to being a marketing buff, I’m also laser-focused on processes and experiences. Creating a great customer experience doesn’t have to mean going all out—it simply starts with doing what you say you’re going to do when you say you’ll do it. (It’s also about great customer service.)

93% of customers say good service makes them more likely to return, and 80% say the experience is as important as a company’s products or services.

Reputation management software allows you to read the room — especially when you’re not in it so you can get hints about what people are saying about your business and plug any holes that appear or adapt based on what your audience wants.

In fact, unhappy customers present one of the best opportunities to learn and improve. So, even though good reviews are important, your response to negative feedback can also help build trust.

With so many options out there, choosing the right review management software can be a real struggle. Here's how you can zero in on the right review management software for your business:

First, I recommend taking some time to identify what you need. A good place to start is by considering questions like these:

• How many reviews will you need to manage and respond to on a daily, weekly, or monthly basis?
• What are the review platforms that are most relevant to your business (e.g., Google, Facebook, Yelp, TripAdvisor)?
• What features are important to you (review collection, response management, analytics, sentiment analysis, and integration capabilities)?
• What investment am I comfortable making?

Then, make a list of the different review management software options and the features they offer and compare them to your needs.

A few things to consider when you compare:

• Pricing. Weigh the cost vs. value provided by each option, and be aware of costs like setup fees, additional charges for premium features, or per-user fees.
• Ease of use. The software should be intuitive and easy to navigate for all team members.
• Scalability. Ensure the software will still work for your business as it grows.
• Reviews and case studies. Check review sites like G2, Capterra, and Trustpilot to see what other users are saying about the software, and look for reviews from businesses similar to yours. Many software providers have case studies or testimonials on their websites which can help you learn how the software is used and has helped other businesses.
• Integration and compatibility. Find out if the software integrates with your existing systems, such as your CRM, email marketing, and social media management tools. I also recommend checking if the software offers API access for custom integrations.
• Support and training. Your review management software should have excellent customer support and make sure you have what you need to start out on the right foot. They should offer onboarding and training resources such as tutorials, webinars, and documentation.

Next, it‘s time to test the software. There’s no better way to find out if the software will work for you than to take advantage of free trials and demos so you can experience it first-hand.

At this point, if you‘ve landed on an option that checks all the right boxes, you can move forward with signing up, knowing that you’ve made the right choice.

By taking a methodical approach, you can save yourself a ton of headaches (that I've seen many business owners suffer) by being too hasty.

Try HubSpot’s Customer Feedback Software to get a better understanding of what they want and think so you can wow them. Get a demo today .

Don't forget to share this post!

Related articles.

I Tested the Best AI Customer Service Software, Here’s What I Found

How to Respond to Customer Complaints [+Complaint Response Examples]

Tool Sprawl in CX: 7 Signs It's Hurting Your Work & 6 Ways to Fix It

25 Testimonial Examples to Build Hype

5 Expert Tips for Responding to Customer Reviews [+ Examples]

The 12-Step Guide to Creating High-Converting Customer Testimonial Videos

How to Delete Google Reviews (& What to Do If You Can't)

33 Customer Review Sites for Collecting Business & Product Reviews

Why Paying for Google Reviews Is a No-Go (+ What To Do Instead)

Google Review Link: How to Generate One and Share

Truly understand your customers and spot opportunities to make them happier.

Service Hub provides everything you need to delight and retain customers while supporting the success of your whole front office

Get started

• Project management
• CRM and Sales
• Work management
• Product development life cycle
• Comparisons
• Construction management
• monday.com updates

19 Best Project Management Software for 2024

Welcome to your roadmap for project management success. We’ve reviewed the market’s top tools to bring you this concise, powerful guide. Our curated list delivers key insights on each software’s features, strengths, limitations, and pricing. Whether you’re scaling complex projects or streamlining daily tasks, you’ll find the project management software to fit your needs.

We’ll also explore the versatility of monday work management and how it can transform the way your team works.

What is project management software?

Project management software includes all the platforms and tools that help managers and teams plan, coordinate, and execute every aspect of project planning.

There’s an incredibly wide variety of project management software— from personal to-do lists to comprehensive “all-in-one” solutions —that facilitates the production of work for businesses of any shape and size and defines your overall project management methodology.

Read how to successfully manage projects in our complete  project management guide .

What to look for in a project management tool

One of the main benefits of good project management software is that it should serve as an all-in-one tool and provides these essential features:

1. Seamless team collaboration and shared task lists

A project management software should have features that enable project team members to communicate and collaborate effectively such as document sharing, communication in real-time (notes, comments, files) and activity feeds.

2. Gantt charts and timeline visualization

Gantt charts and timeline visualization help in planning and tracking project progress, allowing you to identify bottlenecks and make necessary adjustments.

3. Resource Management

Ensure that the right people are assigned to the right task at the right time by looking out for features such as resource planning and tracking.

4. Reporting and Analytics

Integrated analytics tools can help you gain insights into project progress , team performance, cost and other relevant metrics to make data-driven decisions.

5. Number of users

Effective project management software solutions should be able to accommodate teams of various sizes, from small businesses to large enterprises. Many solutions offer scalability to support growing organizations. Whether you’re looking for unlimited users or a minimum number of users per month, there is a project planning software for your needs.

Project management tools typically offer different pricing tiers to cater to various team sizes and needs. Options may include free plans for individuals or small teams, as well as more comprehensive packages for larger organizations or those requiring advanced features.

7. Scalability

Good project management software should be able to grow with an organization, allowing for the addition of users, creation of new projects, and expansion of functionality without significant performance issues.

8. Integrations

Integration capabilities with other commonly used business tools and services are important for streamlining workflows. This may include connections to communication platforms, cloud storage services, and productivity suites.

9. Ease of use

User-friendly interfaces and intuitive design are crucial for widespread adoption within an organization. Features like drag-and-drop functionality, customizable views, and minimal learning curves contribute to ease of use.

10. Additional features

Common features in project management software include:

• Customizable dashboards for real-time visibility into project progress
• Support for both individual tasks and unlimited tasks to scale with your needs
• Flexible task boards adaptable to Agile workflows and complex workflows
• Advanced task assignment and tracking capabilities across unlimited projects
• Robust file sharing and collaboration tools to enhance team productivity
• Automation capabilities to streamline repetitive processes
• Time tracking and workload management for optimal resource allocation

These features work together to provide a comprehensive project management solution, enabling teams to efficiently plan, execute, and monitor their projects regardless of size or complexity.

How to choose project management software?

The project management software market offers numerous options, from basic plans to comprehensive solutions. Your choice should align with your team’s unique needs and workflows.

Some common factors that teams and companies focus on are ease-of-use, scalability, reporting, security, and pricing. 

Finding the right fit may require trying out different project management software options. Many providers offer free trials, allowing you to test-drive their features before committing.

Ultimately, the best solution is one that your team embraces and that enhances your project management processes. Take the time to explore and compare to find the tool that will drive your team’s success.

Top 19 project management software

Let’s kick off with a snapshot of the top project management tools, ranked based on their capabilities, reporting features, support, security, and value for money:Let’s kick off with a snapshot of the top project management tools, ranked based on their capabilities, reporting features, support, security, and value for money:

1. monday work management

Built upon the monday.com Work OS,  monday work management is designed to help teams work without limits when it comes to project and task management . monday work management offers a user-friendly and intuitive interface with a range of customizable features that cater to the specific collaboration and communication needs of any team across industries.

Top features of monday work management

Project portfolio management

With the new project portfolio solution you can and get a birds-eye view on project progress of multiple projects. The visibility can help you work through any project challenge with ease.

Gantt charts monday.com Gantt chart view allows you to create task dependencies and update task owners and due dates with ease.

Workflow builder Choose from 100s of pre-made automation recipes or create your own custom ones to create workflows that sync integrations, people, and more.

• Best-in-class customer support
• AI features
• Excellent mobile app for on-the-go project management

monday work management pricing:

• Free: $0 for up to 2 users
• Basic: $9/month/user with annual billing
• Standard: $12/month/user with annual billing
• Pro: $19/month/user with annual billing
• Enterprise: Contact sales for pricing

More details on monday.com pricing.

Why customers love monday.com

• Trustradius: 8.5 out of 10 Voted in 2023: best features set, relationship and value
• G2: 4.7 out of 5 Voted in 2023: Leader in over 18 categories
• Capterra: 4.6 out of 5 Shortlisted in 2023 in over 8 software categories

Integrations: monday work management offers over 200+ integrations, including popular tools like Slack, Google Drive, Microsoft Teams, Zoom, and Salesforce.

It also has a robust API for custom integrations.

Asana is a popular project management tool designed to help teams collaborate and manage their tasks and projects more efficiently. Asana has gained a reputation as a reliable and easy-to-use tool for project management, and it is used by many businesses and organizations around the world.

Asana top features

Calendar view: See all of your most important deadlines in one dedicated calendar.

Agile framework: The software has a time boxed, iterative approach for product development workflows.

Change requests and case management : Enable efficient tracking of customer requests with priority and timeline.

• Manage all project documentation in one place.
• Asana also uses drag-and-drop for ease of use.
• Asana may not be suited for larger teams or those who don’t prefer an agile approach.
• Limited customization options, such as notifications.

Asana pricing: $10.99/user/month with annual plan. Free plan available. More details on Asana pricing and review here.

Integrations: Asana integrates with over 200 apps, including Slack, Google Drive, Microsoft Office 365, Dropbox, and Salesforce. It also offers a flexible API for custom integrations.

“Asana is excellent for managing a project involving multiple workers. It provides the team with increased visibility by displaying updates and working on a single thread. Asana is excellent for straightforward project management. Some advanced project management applications, such as Gantt charts, need paid subscriptions, however Asana excels at establishing and keeping classified task lists, as well as allocating tasks to specific individuals.” — G2 Review

Comparison: explore this Asana alternative .

Trello  is a web-based project management tool that uses a board-based approach to help individuals and teams organize their tasks and projects. It is a popular and user-friendly tool that allows users to easily track their progress, collaborate with team members, and visualize their workflow simply and intuitively.

Trello top features

Kanban cards: Trello uses Kanban cards to help you track task and project progress.

Checklists: Break down cards into smaller tasks by adding checklists to them.

Activity feed: See all recent changes and activity on your project board, like card updates and comments.

• It’s a great tool for teams who prefer iterations and Kanban.
• Teams can communicate in real time.
• Trello doesn’t offer project management software classics like Gantt charts or resource management features.
• Kanban cards might not be ideal for larger, complex, and unpredictable projects like construction projects.

Trello Pricing: $5/user/month with annual plan. Free plan available. More details on Trello pricing and review here .

Integrations: Trello offers numerous “Power-Ups” which are essentially integrations. These include popular tools like Slack, Google Drive, Jira, and Salesforce. It also has an API for custom integrations.

Reviews: “Trello is a great tool for the strategic management of projects. I like the fact that it is so simple to move tasks from one stage to another by just dragging and dropping them. The visual layout is my assistant in tracking everything at a high level on one screen. In the planning sessions, data presentation is simple as there are different ways of looking at it. It is comprehensible, hence the whole team can easily stay on the same page.” — G2 Review

Comparison: explore these Trello alternatives & our Trello vs. monday.com guide.

4. Smartsheet

Smartsheet is a software as a service platform to plan, capture, track, automate, and report on work at scale. It offers a rich set of views, reports, workflows, and dashboards to adapt according to business needs.

Smartsheet top features

Dynamic gantt charts: Smartsheet’s Gantt chart view allows users to visualize project timelines and dependencies, with the ability to easily adjust schedules and assignments.

Customizable forms: Smartsheet’s customizable forms feature enables users to gather and organize data, such as customer feedback or project requests.

Data integration: It seamlessly integrates with a wide range of third-party tools, such as Salesforce, Jira, and Microsoft Office.

• Versatile platform:Smartsheet can be used for a wide range of tasks and projects, making it a valuable tool for businesses of all sizes and industries.
• Real-time collaboration:Smartsheet’s collaborative features allow team members to work together in real-time, improving communication and productivity.
• Reporting:The reporting functionality could be improved, with limited options for creating custom reports.
• Missed add-ons:It requires add-ons for time tracking and resource management.

Integrations: Smartsheet integrates with a wide range of business tools, including Microsoft 365, Google Workspace, Salesforce, Jira, and Slack. It also provides connectors for custom integrations.

Reviews: “Smartsheet exceeded my expectations when used for phase gate and readiness checklists, or when organizing projects and assigning stakeholders. It is far more featured out of the box than Excel, and has a place if a project management app like Asana is overkill.” — G2 Review

Smartsheet pricing: $7/user/month with annual plan. Free plan available. More details on Smartsheet pricing and review here.

Jira Software is a server-based customizable workflow management solution that organizes tasks and projects into a centralized platform. It provides businesses with an infrastructure for automated processes and increased productivity.

Jira top features

Configuration: This software is known to be   highly configurable and customizable with granular control over security, privacy, and workflows.

Timeline view: Backed by Gantt Chart, timeline view lets users map dependencies and plans work effectively.

Cross-project syncing: You can mark tasks as a duplicate across projects to keep track of work between teams.

• Subtasks: Break up a task into smaller parts, or show additional steps to complete an overall task.
• Due dates: Track important dates and time so everyone’s working off the same deadline—no matter their time zone.
• Limited pre-made templates:Only 23 pre-made templates are available, which can be a challenge.
• Lack of integrations:Jira Core lacks integrations with many other productivity and task management sharing tools and apps.

Jira pricing: $5/user/month with annual plan. Free plan available. More details on Jira review and pricing here.

Integrations: Jira offers a vast marketplace of apps and integrations, including connections with tools like Slack, GitHub, Bitbucket, and Zendesk. It also has a robust API for custom integrations.

Reviews: “Jira has been a game-changer for my team’s project management. The best thing about this is its issues and tracking features which helps us to deliver features on time and helps us stay ahead. Also, since it integrates with other tools, like Confluence, github etc, it helps us to track the changes and view the updates without even leaving the JIRA board.” — G2 Review

Comparison: explore these Jira alternatives

Wrike is a cloud-based project management software designed to help teams streamline their workflows and collaborate more effectively. It was founded in 2006 by Andrew Filev and has since grown into a popular tool used by businesses of all sizes to manage projects from start to finish.

Wrike top features

Integrations: Wrike integrates with project management must-haves like Google Drive, Zendesk, WordPress, Zapier, Slack, and more.

Wrike Approvals: The software has a dedicated feature to make the project review process smoother.

Budget & expense management: Customize currency preferences and create default hourly rates and job roles for projects.

• Create dashboards for visual and accurate stakeholder communications.
• There is a high level of customization possibilities.
• Some users had hoped for more triggers and actions available under their automation engine.
• You have to pay extra for advanced reporting.

Wrike pricing: $9.80/user/month with annual plan. Free plan available. More details about Wrike pricing and review  here.

Integrations: Wrike integrates with over 400 apps, including popular tools like Salesforce, Microsoft Teams, Google Drive, and Adobe Creative Cloud. It also offers an open API for custom integrations.

Reviews: “Wrike is a great tool for project managment! Wrike keeps me on track, organized, transparent, and saves me time. I use it to view my projects and tasks for the day, week, and upcoming months, and communicate internally with stakeholders, colleages and project managers.” — G2 Review

7. Basecamp

Basecamp is a project management tool that helps teams to   stay organized, collaborate efficiently, and complete projects on time. It was first launched in 2004 by 37Signals, but later rebranded as Basecamp in 2014. Basecamp is a cloud-based project management software that can be accessed through a web browser, desktop, or mobile application.

Basecamp top features

Hill charts: Basecamp uses unique hill charts to give you an overview of project progress.

Automated check-ins: Create routine reminders for team members to provide project updates.

File storage: Keep project documents and assets organized in the same place you manage projects.

• Basecamp is configured for communication with message boards and more.
• This software works well for small to medium-sized teams.
• There aren’t specific features for invoicing.
• Some users said it’s difficult to tag tasks with priority level or other attributes.

Pricing:  $15/user/month with annual plan. Free plan unavailable. More info on  Basecamp pricing  here.

Integrations: Basecamp offers integrations with tools like Slack, Google Drive, and Zapier. While its native integrations are more limited compared to some competitors, it does offer an API for custom solutions.

Reviews: “Project management is a lot more effective now that I use Basecamp for business. The entire team is kept up to date on given tasks, due dates, and relevant developments in real time. Additionally, Basecamp provides a number of file organisation and sharing capabilities, which greatly facilitates teamwork.” — G2 Review

Comparison: explore these Basecamp alternatives

ClickUp is a cloud based collaboration and project management platform designed to streamline team workflows. It offers a wide range of features such as custom statuses, time tracking, and reporting.

Clickup top features

Multitask toolbar: The multitask toolbar allows users to edit multiple tasks simultaneously, saving time and effort.

Custom fields: ClickUp’s custom fields enable it to tailor the platform to your specific workflow, making it more efficient and effective.

Mind Maps: ClickUp’s mind maps feature enables users to visually map out their ideas and workflows, making it easy to see how tasks relate to one another and streamline the project planning process.

• Agile project management : ClickUp’s agile project management tools help users keep track of team’s progress and adjust workflow as needed.
• Goals: Users can set, track, and measure progress towards their objectives, ensuring that everyone is aligned and working towards the same goals.
• Inefficient comment’s threads: Comment’s threads and replays at chat space are hard to follow, they show entangled or not easy to find at a glance.
• Non-responsive dashboard:While dashboards are very nice, they are extremely slow to update and refresh.

Pricing: Free version is available for personal usage with limited features and paid plan starts from $7 per member per month

Integrations: ClickUp offers over 1,000+ integrations through native connections and its integration with Zapier. Popular integrations include Slack, Google Drive, GitHub, and Zoom.

Reviews: “ClickUp is easy to use. ClickUp has become a business tool that I cannot do without. I have both a business and a personal account because I adore this app. There are other task and project management choices available, however, none compare to ClickUp in terms of customization or functionality. They provide best customer service.” — G2 Review

Comparison: explore these ClickUp alternatives and our ClickUp vs monday.com comparison guide.

9. Zoho Projects

Zoho Projects is a cloud based project management software that helps organizations manage their resources. It comes with tools for task management, reporting, and issue tracking .

Zoho projects top features

Blueprint: Zoho Projects’ Blueprint feature allows users to create project templates, making it easier to launch new projects with predefined workflows. Risk management: The Risk Management feature helps users identify and manage potential project risks, minimizing the chance of project delays or failures. Custom fields: With custom fields, users can track and report unique project metrics specific to their needs

• Comprehensive analysis: Comes with a suite of reporting tools to gain insights into project performance and make data-driven decisions.
• Workflows: You can automate routine tasks and processes, allowing teams to focus on high-value workflows.
• Limited integrations: Zoho’s integrations for leave management system and attendance are very limited.
• Document storage: The software’s document storage function isn’t compatible for every file type.

Pricing:  $4/user/month with annual plan. Free plan available.

Integrations: Zoho Projects integrates seamlessly with other Zoho apps and offers integrations with popular third-party tools like Google Apps, Microsoft Office 365, and Slack. It also provides an API for custom integrations.

Reviews: “I love the interface of Zoho, it is very intuitive and very easy to use. It has a lot of features task management, people management, time tracking. Also, there are collaboration features as well. The dashboard for every single project helps me a lot to keep track of everything. The integration with other apps is also very useful.” — G2 Review

Nifty Project Management is an online project management tool designed to improve team collaboration and project efficiency.

Nifty top features

Task management: Team members can create, assign, and track tasks, set deadlines, and receive notifications.

Time tracking: A built-in time-tracking feature helps team members track their time spent on specific tasks.

Project roadmaps: Plot milestones and deadlines.

• Project portfolio management: Nifty lets you create folders for projects into portfolios based on operations, account ownership, client delivery, and more.
• Time tracking: The built-in time tracking feature helps teams keep track of their work hours and make more accurate estimates for future projects.
• Limited customization: Nifty’s customization options are limited compared to other project management tools.
• Lack of advanced features: Nifty may not have some of the advanced features that larger teams or more complex projects may require.

Pricing: $3.90/user/month with annual plan. Free plan available.

Integrations: Nifty offers integrations with tools like Slack, Google Drive, Zoom, and GitHub. It also integrates with Zapier, expanding its connectivity options.

Reviews: “Nifty is simple to use, but at the same time has a lot of features. It’s intuitive, so no need to get a whole training to be able to use it. In the company, we use it mainly for project management, task assignment, docs organization, and client communication. Even the less tech-savvy client is able to use it with no major issues.” — G2 Review

11. Todoist

Todoist is a web-based project management and productivity platform designed to schedule and plan daily routine tasks. It lets users collect tasks, organize projects and plan their day.

Todoist top features

Natural language input: Todoist allows users to add tasks quickly and easily using natural language, without the need for complicated forms or menus.

Smart schedule: Todoist’s smart schedule feature suggests the best date and time to schedule tasks based on their due dates, priorities, and other factors.

Customization: Todoist offers a range of customization options, including themes, filters, and labels, allowing users to personalize their task management experience.

• Remote access: Todoist is available on multiple platforms, including web, mobile, and desktop, making it easy to access and manage tasks from anywhere.
• Reminders:Reminders feature sends notifications to users when a task is due or approaching its due date.
• Complexity:Some users may find Todoist’s interface and features overwhelming, especially if they are new to task management apps.
• No-offline access: Todoist requires an internet connection, which could be problematic for users who need offline access.

Pricing: $4/user/month with annual plan. Free plan available.

Integrations: Todoist integrates with a variety of productivity tools including Google Calendar, Slack, Zapier, and Amazon Alexa. It also offers an API for custom integrations.

Reviews: “For me, Todoist is like a lifesaver. It was easy to set up and really helps in putting everything into clear to-do lists, even for big projects with my team. Plus, it makes changing deadlines very easy.” — G2 Review

Miro is a powerful project management and collaboration tool that helps teams work more efficiently and effectively. With its user-friendly interface and comprehensive features, Miro allows teams to plan, execute, and deliver projects.

Miro features

Visual collaboration: Users can create boards, add sticky notes, and invite team members to work together from anywhere in the world.

Integrations: Miro integrates with a variety of popular tools like Trello, Slack, and Google Drive.

Templates: Miro provides a library of templates for different types of projects, including product roadmaps, user story maps, and agile boards.

• Customizable: Miro can be customized to fit the specific needs of a team or project.
• Real-time collaboration: Miro’s mobile app allows teams to work together in real-time, no matter where they are located.
• Miro can be expensive for smaller teams and individual users.
• There can be a learning curve for some users, especially those who are not familiar with visual collaboration tools.

Pricing:  $8/user/month with annual plan. Free plan available.

Integrations: Miro offers integrations with over 100 tools including Slack, Microsoft Teams, Asana, Jira, and Google Drive. It also provides an API for custom integrations.

Reviews: “I absolutely love the overall interface of Miro. It’s clean, inviting, and aesthetically pleasing, making it a joy to use. The user-friendly design ensures that even complex tasks are easy to manage, which significantly enhances productivity and collaboration.” — G2 Review

Hive is a productivity platform designed for teams to collaborate and streamline their workflow. It offers real-time communication, task management, and analytics to ensure teams stay on track and meet their goals.

Hive top features

Action templates: Allows users to create and automate tasks, saving time and increasing efficiency.

Forms: Users can create custom forms to collect data and feedback from team members and stakeholders.

External actions: There is an option to integrate with external tools like Dropbox, Google Drive, and Salesforce.

• Customizable workspace: Users can personalize their workspace to fit their needs and preferences.
• Robust analytics: Provides detailed insights into team performance and project progress.
• Limited integrations: Compared to other project management tools, Hive has a limited number of integrations available.
• Limited mobile app functionality: The mobile app lacks some of the features available on the desktop version.

Pricing:  $5/user/month with annual plan. Free plan available.

Integrations: Hive integrates with over 1,000 apps through native integrations and Zapier. Popular integrations include Slack, Zoom, Google Drive, and Salesforce.

Reviews: “It is intuitive, easy to use, and what you can’t figure out? They have great customer support and the Hive Univeristy. We use it daily. Like in daily task lists, the templates are AWESOME!. They’ve made things so easy for us to set up programs, consistent habits, build projects, communicate, it’s amazing!” — G2 Review

Notion is a web based productivity and note-taking app comprising several organizational tools such as task management, to-do lists, project tracking and bookmarking.

Notion top features

Embeddable content: Notion allows users to embed various types of content such as Google Docs, Trello boards, and more, enhancing collaboration and accessibility.

Relational databases: With Notion’s relational database feature, users can connect different types of data across multiple pages and databases, making it easier to track information and analyze data.

Customizable templates: Notion provides a vast selection of customizable templates.

• Powerful integrations: Notion integrates with Trello, Slack, and Google Drive, and more.
• Offline access: Notion allows users to work offline, ensuring that they can stay productive even when they don’t have an internet connection.
• Steep learning curve: Notion’s powerful features can take some time to fully understand and master, which may be a barrier to entry for some users.
• Multi-project management: Some users have reported that managing multi-projects on Notion is a very tedious process.

Pricing: $8/user/month with annual plan. Free plan available.

Integrations: Notion offers integrations with tools like Slack, Google Drive, and Trello. While its native integrations are more limited, it offers an API for custom integrations and connects with Zapier for expanded options.

Reviews: “Notion is incredible at giving you the flexibility you need to create whatever you need—from a simple notes library to reminders to project management. Over my two years of using it I’ve developed systems that help me in my day-to-day tasks both as a business owner, creative freelancer, and even a father of 4 kids. All of my notes and projects live inside Notion and I use it daily.” — G2 Review

Comparison: Explore these Notion alternatives

15. Airtable

Airtable is a web-based low-code productivity platform to build collaborative apps. It’s designed for custom workflow creation, collaboration, and communication on shared development projects.

Airtable top features

Custom extensions: With its Blocks SDK, users can create their own integrations, visualizations, and internal tools.

Linked records: Airtable’s Linked Records feature enables users to connect records from multiple tables within a single base, creating complex relational databases that are easy to manage.

Automations: With Airtable’s Automations, users can automate repetitive tasks and workflows without the need for complex coding or integration.

• Real-time data accessibility: Airtable functions like a relational database where teams can see centralized data as it changes or gets updated in real-time.
• Intuitive apps builder: Airtable’s no-code/low-code architecture allows users to build the tools that meet their needs.
• Limited reporting: Airtable’s reporting capabilities are limited compared to more advanced database software.
• Limited exporting options: Airtable’s exporting options are limited compared to more advanced database software.

Pricing:  $20/user/month with annual plan. Free plan available.

Integrations: Airtable offers a wide range of integrations including Slack, Google Drive, Jira, and Salesforce. It also provides an API and supports Zapier for additional integration options.

Reviews: “Airtable stands out because you can customize everything: fields, views, even templates. This flexibility lets you use it for anything from data visualization to project tracking. Plus, it’s super collaborative – assign tasks, leave comments, and see updates in real time. It’s our daily go-to task tracker, easy to use, and even connects with Google Sheets for extra power.” — G2 Review

Comparison: Explore these Airtable alternatives

16. Adobe Workfront

Adobe Workfront is a comprehensive work management platform designed to help enterprise teams plan, manage, and execute complex projects. It offers robust features for project management, resource allocation, and collaboration, making it suitable for large organizations with diverse project needs.

Adobe Workfront top features

Resource management: Optimize team workload and capacity planning.

Custom workflows: Create and automate workflows tailored to your organization’s needs.

Reporting and analytics: Generate detailed reports and dashboards for data-driven decision-making.

• Highly customizable to fit various business processes.
• Strong integration with other Adobe products.
• Can be complex to set up and may require extensive training.
• Higher price point compared to some alternatives.

Pricing: Custom pricing based on organizational needs. Contact Adobe for a quote.

Integrations: Adobe Workfront offers extensive integrations with Adobe Creative Cloud apps and other popular tools like Slack, Microsoft Teams, and Salesforce. It also provides an API for custom integrations.

Reviews: “I find the assigning tasks and getting notifications helpful, this makes it easy to move through a project or initiative without needing to baby sit people. Having a request queue has been a hige help for my department as well, so we can field requested work coming in and prioritize it before its assigned.” — G2 Review

Comparison: Explore these Adobe Workfront Alternatives

17. Microsoft Project

Microsoft Project is a project management software developed and sold by Microsoft. It’s designed to assist project managers in developing plans, assigning resources, tracking progress, managing budgets, and analyzing workloads.

Microsoft Project top features

Gantt charts: Visualize project timelines and dependencies.

Resource management: Allocate and track resources across projects.

Integration with Office 365: Seamlessly work with other Microsoft tools

• Robust planning and scheduling capabilities.
• Familiar interface for users of other Microsoft products.
• Steep learning curve for new users.
• Limited collaboration features compared to some modern alternatives.

Pricing: Starts at $10/user/month for the cloud-based version. On-premises versions also available.

Integrations: Microsoft Project integrates seamlessly with other Microsoft 365 apps and offers some third-party integrations. It also provides an API for custom integrations, though its integration ecosystem is more limited compared to some competitors.

Reviews: “It’s easy to use, can be tempalitized, easy to export and maintain the critical paths. Also it’s integration with Microsoft and few Non Microsoft products. Its better product for sml to mid size organisations to manage the project and monitor it’s progress.” — G2 Review

18. ActiveCollab

ActiveCollab is a project management tool that focuses on simplicity and ease of use. It’s designed to help teams organize tasks, collaborate on projects, and track time and expenses.

ActiveCollab top features

Task management: Create, assign, and track tasks with ease.

Time tracking: Built-in timer for accurate project time tracking.

Invoicing: Generate invoices based on tracked time and expenses.

• User-friendly interface with a short learning curve.
• Combines project management with basic invoicing features.
• May lack some advanced features needed by larger teams.
• Limited integration options compared to some competitors.

Pricing: Starts at $7.50/user/month when billed annually. Self-hosted option available.

Integrations: ActiveCollab offers integrations with tools like Slack, Zapier, and QuickBooks. While its native integrations are more limited, it does provide an API for custom solutions.

Reviews: “It’s an ideal tool for tracking project management activities during our weekly lineouts. In my sprint, I can see what everyone is working on, write notes, track time, and label it properly.” — G2 Review

Height is a modern project management and team collaboration tool that aims to combine the best features of task management, documentation, and communication platforms.

Height top features

Flexible views: Switch between list, board, and calendar views.

Real-time collaboration: Edit and comment on tasks in real-time.

Customizable fields: Tailor task properties to fit your workflow.

• Clean, intuitive interface.
• Combines task management with team chat functionality.
• Relatively new platform, may lack some advanced features.
• Limited integrations compared to more established tools.

Integrations: Height offers integrations with popular tools like Slack, GitHub, and Figma. As a newer platform, its integration ecosystem is still growing, but it does offer an API for custom integrations.

Reviews: “UI is very clear and looks great! Clean colors, visible tasks. I am using a well-known task tool and the interface is making the work unbearable. This tool seems so simple and efficient, I hope it lives up to its promise! I only just started using it, but I have a good feeling this is going to make managing projects easier.” — G2 Review

Additional project management software

While the tools mentioned above are some of the most popular options, the project management software landscape is diverse. Here are some additional tools worth considering, each with its unique strengths:

• GanttPro: Specializes in Gantt chart creation and management with a browser-based interface.
• ProjectManager: Versatile solution with comprehensive features for planning and tracking projects.
• TeamGantt: Focuses on Gantt chart functionality for work visualization and tracking.
• Forecast: Stands out with advanced resource planning capabilities and robust project management features.
• Teamwork.com: Popular among marketing and professional service teams for managing projects, deliverables, and billing.
• Zenhub: Tailored for software development teams with close GitHub integration.
• ProWorkflow: Excels in supporting both internal and external collaboration with customizable workflows.
• Celoxis: Comprehensive platform suitable for complex, enterprise-level projects.
• Plutio: All-in-one tool designed for freelancers and small businesses, combining task management, time tracking, and invoicing.

Why do companies need project management software? 

Project management software transforms how teams work, boosting efficiency and driving success.

Here’s why companies rely on it:

• Enhances productivity: Teams accomplish more in less time by streamlining workflows and automating repetitive tasks.
• Centralizes file and data access: All project-related information is stored in one place, eliminating time wasted searching for documents.
• Simplifies task management: Easily assign, track, and complete tasks, turning complex projects into manageable steps.
• Increases transparency: Team members clearly see who’s responsible for what, reducing confusion and duplicated efforts.
• Enables real-time goal setting and progress tracking: Set clear objectives and monitor progress instantly, allowing for timely adjustments.
• Improves resource management: Efficiently allocate time, skills, and resources across projects to maximize output.
• Boosts collaboration and communication: Facilitates seamless interaction between team members, regardless of location or time zone.

By addressing these key areas, project management software helps companies stay organized, meet deadlines, and achieve their goals more effectively. It’s an essential tool for any business looking to optimize their operations and stay competitive in today’s fast-paced market.

Choose the project management platform that will best fit your business

As more teams work remotely and more businesses see the value of an organized work environment, it’s crucial that you identify which project planning software is right for you.

Do you want something focused on time and task management? Or monday work management, a complete, customizable, open platform that makes work…work.

What is the most commonly used project management tool?

While popularity varies by industry and team size, monday.com is one of the most widely adopted project management tools. Its versatility and user-friendly interface contribute to its widespread use.

What app do project managers use?

Project managers use a variety of tools, with monday.com being a top choice. It offers comprehensive features for planning, tracking, and collaborating, making it suitable for diverse project management needs.

- Enhanced productivity - Centralized file and data access - Simple task management - Transparency between team members - Real-time goal setting and progress tracking - Accurate resource management - Better collaboration and communication

Is there a Google project management tool?

Google offers some basic project management features within Google Workspace, but it's not a dedicated project management solution. For more robust capabilities, many teams turn to specialized tools like monday.com, which also integrates well with Google's suite.

1. Portfolio management view so teams can stay on top of everything 2. Automations to automate aspects of team processes and get time back for the more important things 3. Dashboards to share valuable insight

Which project management software is best for beginners?

monday.com is often recommended for beginners due to its intuitive interface and customizable features. It offers a gentle learning curve while providing powerful project management capabilities that grow with your needs.

What is the most user-friendly project management tool for 2024?

User-friendliness is subjective, but monday.com consistently ranks high in this category. Its visual approach, drag-and-drop functionality, and customizable views make it accessible for users of all experience levels.

TechRepublic

Account information.

Share with Your Friends

8 Best Scrumban Project Management Software for 2024

Your email has been sent

Initially conceived to help teams transition from scrum to kanban, scrumban has evolved into a standalone hybrid project management methodology. As project requirements become more complex and fluid, scrumban’s hybrid approach offers a versatile solution for teams looking for more flexibility.

Here are eight of the best current project management software tools that support scrumban, compared against their key features, pricing and suitability for a variety of team needs.

• Best for software development teams: Jira
• Best for feature variety: ClickUp
• Best for visual collaboration: Miro
• Best for resource management: Teamwork
• Best for flexible scrumban workflows: Asana
• Best for scrumban automation: monday.com
• Best for complex projects: Wrike
• Best for enterprise scrumban management: Rally

Top scrumban project management software comparison

Here’s a comparison of top scrumban project management tools that highlights their ratings, best uses, starting prices and key scrumban features.

Jira: Best for software development teams

Our rating: 4.6 out of 5

Jira by Atlassian is a leading project management tool designed specifically for software development teams practicing scrumban. It seamlessly combines the structured sprint planning of scrum with the continuous flow and visual task management of kanban . This allows software development teams to customize workflows, set WIP limits and manage tasks efficiently. Jira’s robust reporting tools provide critical insights into team performance, making it easier to track progress and make data-driven decisions.

Why I chose Jira

I chose Jira because of its powerful features tailored for software development teams practicing scrumban. Its ability to manage both structured sprints and continuous task flows is essential for maintaining the dynamic nature of software projects.

The integration of kanplan, which blends kanban’s visual task management with scrum’s backlog grooming, offers a unique advantage in keeping projects organized and on track. Additionally, Jira’s extensive reporting tools, such as burndown and velocity charts, deliver valuable insights into team performance and project progress.

• Free: $0 for up to 10 users.
• Standard: $8.15 per user billed monthly and $850 annually for 1–10 user tier.
• Premium: $16 per user billed monthly and $1,600 annually for 1–10 user tier.
• Enterprise: Available for teams with more than 201 users and only billed annually. Contact Jira Service Management sales for pricing information.

Standout features

• Custom workflows to match specific scrumban processes, blending sprint planning with continuous delivery ( Figure A ).
• WIP limits to maintain efficiency and prevent task overload.
• Kanplan to combine kanban’s visual board with scrum’s backlog, allowing for efficient backlog grooming and task management.
• Advanced reporting with burndown charts, velocity charts and cumulative flow diagrams for data-driven retrospectives and performance tracking.

Pros and cons

ClickUp: Best for feature variety

Our rating: 5 out of 5

ClickUp is a versatile project management tool with all sorts of features that support scrumban. It combines more than 15 views with structured sprint planning, which helps teams find that balance between flexible and organized project management. ClickUp’s highly customizable interface and vast feature set means its users can adapt their workflows to match all manner of project requirements, whether long or short-term.

Why I chose ClickUp

I chose ClickUp because of its huge feature catalog. For instance, its multiple views offered me variety for visualizing my workflows effectively, with the option of using its Board view as a scrum board to maintain the structured nature I needed. It also offers a task management kanban template to quickly get off the ground with kanban. ClickUp also delivers customizable task statuses and automation capabilities that streamline task management and improve efficiency.

• Free plan available.
• Unlimited: $7 per user per month if billed annually, or $10 per user per month if billed monthly.
• Business: $12 per user per month if billed annually, or $19 per user per month if billed monthly.
• Enterprise: Custom pricing available for larger organizations.
• Visualize tasks with the kanban Board view, allowing drag-and-drop management and customizable columns ( Figure B ).
• Custom task statuses for better workflow management, tailored to scrumban processes.
• Automation to reduce manual effort with rules for recurring tasks and status changes.
• Advanced reporting and dashboards to track progress and performance.

Miro: Best for visual collaboration

Miro is a dynamic collaboration tool whose versatile visual project management features align with scrumban practices. Pairing the strengths of kanban boards and scrum methodologies, Miro gives users an infinite digital canvas for teams to visually organize tasks, brainstorm ideas and track project progress. This flexibility makes it particularly useful for remote and distributed teams that most appreciate a customizable visual platform to manage their workflows.

Why I chose Miro

Miro caught my eye because of its unique visual project management that flexibly converts sticky notes, shapes and text boxes into cards with information. This delivers an infinite canvas for expansive project mapping and brainstorming without space constraints, fostering creative and engaging collaboration. Plus, I found that Miro’s real-time collaboration and integration with tools like Jira and Slack made it more transparent and efficient for scrumban.

• Free: Basic features for individual users or small teams.
• Starter: $8 per user per month billed annually or $10 monthly.
• Business: $16 per user per month billed annually or $20 monthly.
• Enterprise: Custom pricing available starting from 30 plus members.
• Infinite canvas for limitless project mapping and brainstorming.
• Real-time collaboration to facilitate instant teamwork and efficiency.
• Customizable scrum templates and kanban boards ( Figure C ).
• Interactive brainstorming tools like mind maps and sticky notes.

Teamwork: Best for resource management

Our rating: 3.7 out of 5

Teamwork is a robust project management tool with support for scrumban through detailed resource management capabilities. It enables teams to plan, track and optimize resource allocation with features like task dependencies, time tracking and workload management. Teamwork is also capable of visualizing workflows through customizable kanban boards and advanced reporting tools, which come in handy when managing resources across multiple projects and ensuring transparency.

Why I chose Teamwork

I chose Teamwork for its resource management as it best visualized resource allocation and tracked workloads, which helped me prevent bottlenecks in my project. With Teamwork, I can easily balance workloads, reassign tasks and monitor project progress to maintain a steady flow in scrumban projects.

• Free: $0 for up to 5 users.
• Deliver: $11 per user per month if billed annually, or $13.99 per user per month if billed monthly.
• Grow: $19.99 per user per month if billed annually, or $25.99 per user per month if billed monthly.
• Scale: Custom pricing available for larger organizations.
• Customizable kanban boards for visual task management and workflow optimization ( Figure D ).
• Sprint planning tools to help organize and execute sprints efficiently.
• Advanced reporting features for insightful project performance metrics.
• Time tracking and resource management to ensure optimal task allocation.

Asana: Best for flexible scrumban workflows

Our rating: 3.9 out of 5

Asana is a versatile project management tool that aligns with scrumban by providing a flexible framework for managing tasks and workflows. The platform not only delivers both scrum and kanban software features but also moves seamlessly between these methodologies. Additionally, Asana has a customizable scrumban template and automation features that help teams optimize their workflows and improve productivity.

Why I chose Asana

Asana made it onto my list because of its scrumban-focused customizable template. Many tools support scrumban by providing scrum and kanban capabilities and direction on how to combine them intuitively. However, Asana takes this a step further with a scrumban template that includes directions on how to organize the information. This provides consistency when transitioning from one methodology to the other.

• Basic: Free with limited features.
• Premium: $10.99 per user per month if billed annually, or $13.49 per user per month if billed monthly.
• Business: $24.99 per user per month if billed annually, or $30.49 per user per month if billed monthly.
• Board view for visual task management and workflow optimization ( Figure E ).
• Custom fields to sort and filter work according to team requirements.
• Automation to streamline repetitive tasks and improve efficiency.
• Advanced reporting and dashboards to track project progress and performance.

monday.com: Best for scrumban automation

monday.com is a project management platform whose scrumban qualities are supported by powerful automation and customization capabilities. It allows teams to efficiently manage their tasks, sprints and workflows with features like customizable kanban boards, sprint management and over 150 pre-built automations. Its interface is both attractive and highly customizable, enabling teams to adapt their workflows to all kinds of project needs.

Why I chose monday.com

monday’s advanced automation features simplified and streamlined my scrumban processes through the ability to automate repetitive tasks and reduce manual effort. Its customizable kanban boards and sprint management tools offer a structured yet adaptable framework for managing scrumban projects.

• Free: $0 for up to 2 users.
• Basic: $9 per user per month if billed annually, or $12 billed monthly.
• Standard: $12 per user per month if billed annually, or $14 billed monthly.
• Pro: $19 per user per month if billed annually, or $24 billed monthly.
• Customizable kanban boards to manage tasks and visualize workflows ( Figure F ).
• Sprint management tools for planning and tracking project progress.
• Automation features to streamline repetitive tasks and notifications.
• Advanced reporting and dashboards to monitor project performance.

Wrike: Best for complex projects

Wrike is a powerful project management tool that’s well-suited for complex projects thanks to flexible and robust features for both structured and adaptive workflows. It provides customizable online scrum boards, real-time collaboration tools and advanced reporting capabilities, allowing teams to manage complex projects with ease and transparency. Plus, Wrike’s focus on adaptability and continuous improvement makes it a great choice for teams needing to transition between different project methodologies seamlessly.

Why I chose Wrike

I found Wrike to be set up in the best way to handle complex projects. Its customizable online scrum boards allow teams to visualize their workflows and prioritize tasks, while the real-time collaboration tools enhance communication and teamwork. Additionally, Wrike’s automated reporting and analytics provide valuable insights into project performance, helping teams to continuously improve their processes and adapt to changing project requirements.

• Team: $9.80 per user per month.
• Business: $24.80 per user per month.
• Enterprise: Custom pricing available.
• Pinnacle: Custom pricing available.
• Customizable scrum boards to visualize and manage workflows ( Figure G ).
• Real-time collaboration tools to enhance team communication.
• Automated reporting and analytics for continuous improvement.
• Advanced time tracking and resource management features.

Rally: Best for enterprise scrumban management

Rally, formerly CA Agile Central, is an enterprise-class agile development platform that’s well suited for large-scale scrumban implementations. The platform offers features such as customizable scrum and kanban boards, robust reporting tools and real-time visibility into work in progress. Its project tree and backlog management tools allow teams to visualize and manage tasks effectively.

Why I chose Rally

I saw Rally as a straightforward pick for large-scale agile project management. The platform’s customizable boards and real-time reporting tools provide a clear view of project progress and resource allocation, while its integration with CI/CD pipeline tools like Jenkins and SonarQube makes it more valuable in software development environments.

Contact Broadcom sales for a quote as pricing information isn’t listed publicly.

• Customizable kanban boards for visual task management and workflow optimization.
• Advanced reporting and analytics to monitor project performance ( Figure H ).
• Comprehensive backlog management and project tree visualization.

What is scrumban?

Scrumban is considered to be a hybrid agile project management methodology, which means it’s designed to break down a project into smaller tasks. It combines the visual and flow-focused strengths of kanban with the structure of scrum to enhance scrum’s flexibility. Scrumban is ideal for a team that is looking to work in sprints to produce a large quantity of deliverables.

How do I choose the best scrumban project management software?

To choose the ideal scrumban project management software , you’ll need to determine if it meets the needs of your specific use case. You can assess this by answering questions such as:

• Independent of its agile features, is the tool customizable enough?
• Does it integrate with your existing technology stack?
• Will its automation features help streamline the work you’re doing?
• Does it offer collaboration features that fit your team’s communication style?
• Is this tool scalable if your project needs change?
• How user-friendly is it, and will your team require training?

You also need to evaluate scrumban-specific features to make sure the tool is a good fit for your project needs and enhances your workflow.

Visual workflow management

Look for software that provides kanban boards where you can visualize your work and track the progress of tasks through various stages such as “Backlog,” “In Progress” and “Done.” This feature is useful for understanding the flow of work and identifying bottlenecks in real time​​.

Work-In-Progress limits

Choose tools that allow you to set limits on the number of tasks that can be in progress concurrently. This helps prevent overloading your team and ensures that members focus on completing tasks before taking on new ones. WIP limits ensure a manageable workflow and improve task completion rates​.

Pull system

Opt for software that supports a pull system where team members can pull tasks from the backlog as they have the capacity to work on them, rather than being assigned tasks in advance. With this, you can expect greater flexibility as it allows team members to work at their own pace, which reduces stress and improves productivity​.

Continuous improvement

Select tools that facilitate regular retrospectives and process evaluations, as they help teams identify areas for improvement and make necessary adjustments to workflows. Continuous improvement is a cornerstone of scrumban, which aims to see processes evolve and improve over time​​.

Flexibility

Ensure the software is flexible enough to adjust processes and workflows as needed. This is particularly important in dynamic, fast-paced environments. The tool should support easy modifications to task priorities, workflow stages and project timelines to accommodate shifting project requirements​​.

Methodology

The tools on this list were chosen by first assessing whether they offer agile features and then investigating their agile features to determine their approach to both scrum and kanban. I then tested the tools where possible to get a feel for the kanban and scrum features and compared them to the features they list on their web pages as well as the feedback these tools get from verified user reviews. From these, I was able to have a holistic picture of each tool and determine the strengths, weaknesses and differentiating qualities of each one.

Subscribe to the Project Management Insider Newsletter

Subscribe to Project Management Insider for best practices, reviews and resources. From project scheduling software to project planning apps, stay up to date with the latest in project management tools. Delivered Wednesdays

• Complete Guide to Project Management Methodologies
• The Best Simple Project Management Software
• What Is Project Management?
• 10 Must-Read Books on Project Management
• Best Software For Businesses and End Users

Create a TechRepublic Account

Get the web's best business technology news, tutorials, reviews, trends, and analysis—in your inbox. Let's start with the basics.

* - indicates required fields

Sign in to TechRepublic

Lost your password? Request a new password

Reset Password

Please enter your email adress. You will receive an email message with instructions on how to reset your password.

Check your email for a password reset link. If you didn't receive an email don't forgot to check your spam folder, otherwise contact support .

Welcome. Tell us a little bit about you.

This will help us provide you with customized content.

Want to receive more TechRepublic news?

You're all set.

Thanks for signing up! Keep an eye out for a confirmation email from our team. To ensure any newsletters you subscribed to hit your inbox, make sure to add [email protected] to your contacts list.

• Open access
• Published: 29 July 2024

Predicting hospital length of stay using machine learning on a large open health dataset

• Raunak Jain 1 ,
• Mrityunjai Singh 1 ,
• A. Ravishankar Rao 2 &
• Rahul Garg 1  

BMC Health Services Research volume  24 , Article number:  860 ( 2024 ) Cite this article

27 Accesses

Metrics details

Governments worldwide are facing growing pressure to increase transparency, as citizens demand greater insight into decision-making processes and public spending. An example is the release of open healthcare data to researchers, as healthcare is one of the top economic sectors. Significant information systems development and computational experimentation are required to extract meaning and value from these datasets. We use a large open health dataset provided by the New York State Statewide Planning and Research Cooperative System (SPARCS) containing 2.3 million de-identified patient records. One of the fields in these records is a patient’s length of stay (LoS) in a hospital, which is crucial in estimating healthcare costs and planning hospital capacity for future needs. Hence it would be very beneficial for hospitals to be able to predict the LoS early. The area of machine learning offers a potential solution, which is the focus of the current paper.

We investigated multiple machine learning techniques including feature engineering, regression, and classification trees to predict the length of stay (LoS) of all the hospital procedures currently available in the dataset. Whereas many researchers focus on LoS prediction for a specific disease, a unique feature of our model is its ability to simultaneously handle 285 diagnosis codes from the Clinical Classification System (CCS). We focused on the interpretability and explainability of input features and the resulting models. We developed separate models for newborns and non-newborns.

The study yields promising results, demonstrating the effectiveness of machine learning in predicting LoS. The best R 2 scores achieved are noteworthy: 0.82 for newborns using linear regression and 0.43 for non-newborns using catboost regression. Focusing on cardiovascular disease refines the predictive capability, achieving an improved R 2 score of 0.62. The models not only demonstrate high performance but also provide understandable insights. For instance, birth-weight is employed for predicting LoS in newborns, while diagnostic-related group classification proves valuable for non-newborns.

Our study showcases the practical utility of machine learning models in predicting LoS during patient admittance. The emphasis on interpretability ensures that the models can be easily comprehended and replicated by other researchers. Healthcare stakeholders, including providers, administrators, and patients, stand to benefit significantly. The findings offer valuable insights for cost estimation and capacity planning, contributing to the overall enhancement of healthcare management and delivery.

Peer Review reports

Introduction

Democratic governments worldwide are placing an increasing importance on transparency, as this leads to better governance, market efficiency, improvement, and acceptance of government policies. This is highlighted by reports from the Organization for Economic Co-operation and Development (OECD) an international organization whose mission it is to shape policies that foster prosperity, equality, opportunity and well-being for all [ 1 ]. Openness and transparency have been recognized as pillars for democracy, and also for fostering sustainable development goals [ 2 ], which is a major focus of the United Nations ( https://sustainabledevelopment.un.org/sdg16 ).

An important government function is to provide for the healthcare needs of its citizens. The U.S. spends about $3.6 trillion a year on healthcare, which represents 18% of its GDP [ 3 ]. Other developed nations spend around 10% of their GDP on healthcare. The percentage of GDP spent on healthcare is rising as populations age. Consequently, research on healthcare expenditure and patient outcomes is crucial to maintain viable national economies. It is advantageous for nations to combine investigations by the private sector, government sector, non-profit agencies, and universities to find the best solutions. A promising path is to make health data open, which allows investigators from all sectors to participate and contribute their expertise. Though there are obvious patient privacy concerns, open health data has been made available by organizations such as New York State Statewide Planning and Research Cooperative System (SPARCS) [ 4 ].

Once the data is made available, it needs to be suitably processed to extract meaning and insights that will help healthcare providers and patients. We favor the creation and use of an open-source analytics system so that the entire research community can benefit from the effort [ 5 , 6 , 7 ]. As a concrete demonstration of the utility of our system and approach, we revealed that there is a growing incidence of mental health issues amongst adolescents in specific counties in New York State [ 8 ]. This has resulted in targeted interventions to address these problems in these communities [ 8 ]. Knowing where the problems lie allows policymakers and funding agencies to direct resources where needed.

Healthcare in the U.S. is largely provided through private insurance companies and it is difficult for patients to reliably understand what their expected healthcare costs are [ 9 , 10 ]. It is ironic that consumers can readily find prices of electronics items, books, clothes etc. online, but cannot find information about healthcare as easily. The availability of healthcare information including costs, incidence of diseases, and the expected length of stay for different procedures will allow consumers and patients to make better and more informed choices. For instance, in the U.S., patients can budget pre-tax contributions to health savings accounts, or decide when to opt for an elective surgery based on the expected duration of that procedure.

To achieve this capability, it is essential to have the underlying data and models that interpret the data. Our goal in this paper is twofold: (a) to demonstrate how to design an analytics system that works with open health data and (b) to apply it to a problem of interest to both healthcare providers and patients. Significant advances have been made recently in the fields of data mining, machine-learning and artificial intelligence, with growing applications in healthcare [ 11 ]. To make our work concrete, we use our machine-learning system to predict the length of stay (LoS) in hospitals given the patient information in the open healthcare data released by New York State SPARCS [ 4 ].

The LoS is an important variable in determining healthcare costs, as costs directly increase for longer stays. The analysis by Jones [ 12 ] shows that the trends in LoS, hospital bed capacity and population growth have to be carefully analyzed for capacity planning and to ensure that adequate healthcare can be provided in the future. With certain health conditions such as cardiovascular disease, the hospital LoS is expected to increase due to the aging of the population in many countries worldwide [ 13 ]. During the COVID-19 pandemic, hospital bed capacity became a critical issue [ 14 ], and many regions in the world experienced a shortage of healthcare resources. Hence it is desirable to have models that can predict the LoS for a variety of diseases from available patient data.

The LoS is usually unknown at the time a patient is admitted. Hence, the objective of our research is to investigate whether we can predict the patient LoS from variables collected at the time of admission. By building a predictive model through machine learning techniques, we demonstrate that it is possible to predict the LoS from data that includes the Clinical Classifications Software (CCS) diagnosis code, severity of illness, and the need for surgery. We investigate several analytics techniques including feature selection, feature encoding, feature engineering, model selection, and model training in order to thoroughly explore the choices that affect eventual model performance. By using a linear regression model, we obtain an R 2 value of 0.42 when we predict the LoS from a set of 23 patient features. The success of our model will be beneficial to healthcare providers and policymakers for capacity planning purposes and to understand how to control healthcare costs. Patients and consumers can also use our model to estimate the LoS for procedures they are undergoing or for planning elective surgeries.

Stone et al. [ 15 ] present a survey of techniques used to predict the LoS, which include statistical and arithmetic methods, intelligent data mining approaches and operations-research based methods. Lequertier et al. [ 16 ] surveyed methods for LoS prediction.

The main gap in the literature is that most methods focus on analyzing trends in the LoS or predicting the LoS only for specific conditions or restrict their analysis to data from specific hospitals. For instance, Sridhar et al. [ 17 ] created a model to predict the LoS for joint replacements in rural hospitals in the state of Montana by using a training set with 127 patients and a test set with 31 patients. In contrast, we have developed our model to predict the LoS for 285 different CCS diagnosis codes, over a set of 2.3 million patients over all hospitals in New York state. The CCS diagnosis code refers to the code used by the Clinical Classifications Software system, which encompasses 285 possible diagnosis and procedure categories [ 18 ]. Since the CCS diagnosis codes are too numerous to list, we give a few examples that we analyzed, including but not limited to abdominal hernia, acute myocardial infarction, acute renal failure, behavioral disorders, bladder cancer, Hodgkins disease, multiple sclerosis, multiple myeloma, schizophrenia, septicemia, and varicose veins. To the best of our knowledge, we are not aware of models that predict the LoS on such a variety of diagnosis codes, with a patient sample greater than 2 million records, and with freely available open data. Hence, our investigation is unique from this point of view.

Sotodeh et al. [ 19 ] developed a Markov model to predict the LoS in intensive care unit patients. Ma et al. [ 20 ] used decision tree methods to predict LoS in 11,206 patients with respiratory disease.

Burn et. al. examined trends in the LoS for patients undergoing hip-replacement and knee-replacement in the U.K. [ 21 ]. Their study demonstrated a steady decline in the LoS from 1997–2012. The purpose of their study was to determine factors that contributed to this decline, and they identified improved surgical techniques such as fast-track arthroplasty. However, they did not develop any machine-learning models to predict the LoS.

Hachesu et al. examined the LoS for cardiac disease patients [ 22 ] and found that blood pressure is an important predictor of LoS. Garcia et al. determined factors influencing the LoS for undergoing treatment for hip fracture [ 23 ]. B. Vekaria et al. analyzed the variability of LoS for COVID-19 patients [ 24 ]. Arjannikov et al. [ 25 ] used positive-unlabeled learning to develop a predictive model for LoS.

Gupta et al. [ 26 ] conducted a meta-analysis of previously published papers on the role of nutrition on the LoS of cancer patients, and found that nutrition status is especially important in predicting LoS for gastronintestinal cancer. Similarly, Almashrafi et al. [ 27 ] performed a meta-analysis of existing literature on cardiac patients and reviewed factors affecting their LoS. However, they did not develop quantitative models in their work. Kalgotra et al. [ 28 ] use recurrent neural networks to build a prediction model for LoS.

Daghistani et al. [ 13 ] developed a machine learning model to predict length of stay for cardiac patients. They used a database of 16,414 patient records and predicted the length of stay into three classes, consisting of short LoS (< 3 days), intermediate LoS ( 3–5 days) and long LoS (> 5 days). They used detailed patient information, including blood test results, blood pressure, and patient history including smoking habits. Such detailed information is not available in the much larger SPARCS dataset that we utilized in our study.

Awad et al. [ 29 ] provide a comprehensive review of various techniques to predict the LoS. Though simple statistical methods have been used in the past, they make assumptions that the LoS is normally distributed, whereas the LoS has an exponential distribution [ 29 ]. Consequently, it is preferable to use techniques that do not make assumptions about the distribution of the data. Candidate techniques include regression, classification and regression trees, random forests, and neural networks. Rather than using statistical parametric techniques that fit parameters to specific statistical distributions, we favor data-driven techniques that apply machine-learning.

In 2020, during the height of the COVID-19 pandemic, the Lancet, a premier medical journal drew widespread rebuke [ 30 , 31 , 32 ] for publishing a paper based on questionable data. Many medical journals published expressions of concern [ 33 , 34 ]. The Lancet itself retracted the questionable paper [ 35 ], which is available at [ 36 ] with the stamp “retracted” placed on all pages. One possible solution to prevent such incidents from occurring is for top medical journals to require authors to make their data available for verification by the scientific community. Patient privacy concerns can be mitigated by de-identifying the records made available, as is already done by the New York State SPARCS effort [ 4 ]. Our methodology and analytics system design will become more relevant in the future, as there is a desire to prevent a repetition of the Lancet debacle. Even before the Lancet incident, there was declining trust amongst the public related to medicine and healthcare policy [ 37 ]. This situation continues today, with multiple factors at play, including biased news reporting in mainstream media [ 38 ]. A desirable solution is to make these fields more transparent, by releasing data to the public and explaining the various decisions in terms that the public can understand. The research in this paper demonstrates how such a solution can be developed.

Requirements

We describe the following three requirements of an ideal system for processing open healthcare data

Utilize open-source platforms to permit easy replicability and reproducibility.

Create interpretable and explainable models.

Demonstrate an understanding of how the input features determine the outcomes of interest.

The first requirement captures the need for research to be easily reproduced by peers in the field. There is growing concern that scientific results are becoming hard for researchers to reproduce [ 39 , 40 , 41 ]. This undermines the validity of the research and ultimately hurts the fields. Baker termed this the “reproducibility crisis”, and performed an analysis of the top factors that lead to irreproducibility of research [ 39 ]. Two of the top factors consist of the unavailability of raw data and code.

The second requirement addresses the need for the machine-learning models to produce explanations of their results. Though deep-learning models are popular today, they have been criticized for functioning as black-boxes, and the precise working of the model is hard to discern. In the field of healthcare, it is more desirable to have models that can be explained easily [ 42 ]. Unless healthcare providers understand how a model works, they will be reluctant to apply it in their practice. For instance, Reyes et al. determined that interpretable Artificial Intelligence systems can be better verified, trusted, and adopted in radiology practice [ 43 ].

The third requirement shows that it is important for relevant patient features to be captured that can be related to the outcomes of interest, such as LoS, total cost, mortality rate etc. Furthermore, healthcare providers should be able to understand the influence of these features on the performance of the model [ 44 ]. This is especially critical when feature engineering methods are used to combine existing features and create new features.

In the subsequent sections, we present our design for a healthcare analytics system that satisfies these requirements. We apply this methodology to the specific problem of predicting the LoS.

We have designed the overall system architecture as shown in Fig.  1 . This system is built to handle any open data source. We have shown the New York SPARCS as one of the data sources for the sake of specificity. Our framework can be applied to data from multiple sources such as the Center for Medicare and Medicaid Services (CMS in the U.S.) as shown in our previous work [ 6 ]. We chose a Python-based framework that utilizes Pandas [ 45 ] and Scikit learn [ 46 ]. Python is currently the most popular programming language for engineering and system design applications [ 47 ].

Shows the system architecture. We use Python-based open-source tools such as Pandas and Scikit-Learn to implement the system

In Fig.  2 , we provide a detailed overview of the necessary processing stages. The specific algorithms used in each stage are described in the following sections.

Shows the processing stages in our analytics pipeline

Recent research has shown that it is highly desirable for machine learning models used in the healthcare domain to be explainable to healthcare providers and professionals [ 48 ]. Hence, we focused on the interpretability and explainability of input features in our dataset and the models we chose to explore. We restricted our investigation to models that are explainable, including regression models, multinomial logistic regression, random forests, and decision trees. We also developed separate models for newborns and non-newborns.

Brief description of the dataset

During our investigation, we utilized open-health data provided by the New York State SPARCS system. The data we accessed was from the year 2016, which was the most recent year available at the time. This data was provided in the form of a CSV file, containing 2,343,429 rows and 34 columns. Each row contains de-identified in-patient discharge information. The dataset columns contained various types of information. They included geographic descriptors related to the hospital where care was provided, demographic descriptors such as patient race, ethnicity, and age, medical descriptors such as the CCS diagnosis code, APR DRG code, severity of illness, and length of stay. Additionally, payment descriptors were present, which included information about the type of insurance, total charges, and total cost of the procedure.

Detailed descriptions of all the elements in the data can be found in [ 49 ]. The CCS diagnosis code has been described earlier. The term “DRG” stands for Diagnostic Related Group [ 49 ], which is used by the Center for Medicare and Medicaid services in the U.S. for reimbursement purposes [ 50 ].

The data includes all patients who underwent inpatient procedures at all New York State Hospitals [ 51 ]. The payment for the care can come from multiple sources: Department of Corrections, Federal/State/Local/Veterans Administration, Managed Care, Medicare, Medicaid, Miscellaneous, Private Health Insurance, and Self-Pay. The dataset sourced from the New York State SPARCS system, encompassing a wider patient population beyond Medicare/Medicaid, holds greater value compared to datasets exclusively composed of Medicare/Medicaid patients. For instance, Gilmore et al. analyzed only Medicare patients [ 52 ].

We examine the distribution of the LoS in the dataset, as shown in Fig.  3 . We note that the providers of the data have truncated the length of stay to 120 days. This explains the peak we see at the tail of the distribution.

Distribution of the length of stay in the dataset

Data pre-processing and cleaning

We identified 36,280 samples, comprising 1.55% of the data where there were missing values. These were discarded for further analysis. We removed samples which have Type of Admission = ‘Unknown’ (0.02% samples). So, the final data set has 2,306,668 samples. ‘Payment Typology 2’, and ‘Payment Typology 3’, have missing values (> = 50% samples), which were replaced by a ‘None’ string.

We note that approximately 10% of the dataset consists of rows representing newborns. We treat this group as a separate category. We found that the ‘Birth Weight’ feature had a zero value for non-newborn samples. Accordingly, to better use the ‘Birth Weight’ feature, we partitioned the data into two classes: newborns and non-newborns. This results in two classes of models, one for newborns and the second for all other patients. We removed the ‘Birth Weight’ feature in the input for the non-newborn samples as its value was zero for those samples.

The column ‘Total Costs’ (and in a similar way, ‘Total Charges’) are usually proportional to the LoS, and it would not be fair to use these variables to predict the LoS. Hence, we removed this column. We found that the columns 'Discharge Year', 'Abortion Edit Indicator'' are redundant for LoS prediction models, and we removed them. We also removed the columns ‘CCS Diagnosis Description’, ‘CCS Procedure Description’, ‘APR DRG Description’, ‘APR MDC Description’, and ‘APR Severity of Illness Description’ as we were given their corresponding numerical codes as features.

Since the focus of this paper is on the prediction of the LoS, we analyzed the distribution of LoS values in the dataset.

We developed regression models using all the LoS values, from 1–120. We also developed classification models where we discretized the LoS into specific bins. Since the distribution of LoS values is not uniform, and is heavily clustered around smaller values, we discretized the LoS into a small number of bins, e.g. 6 to 8 bins.

We utilized 10% of the data as a holdout test-set, which was not seen during the training phase. For the remaining 90% of the data, we used tenfold cross-validation in order to train the model and determine the best parameters to use.

Feature encoding

Many variables in the dataset are categorical, e.g., the variable “APR Severity of Illness Description” has the values in the set [Major, Minor, Moderate, Extreme]. We used distribution-dependent target encoding techniques and one-hot techniques to improve the model performance [ 53 ]. We replaced categorical data with the product of mean LoS and median LoS for a category value. The categorical feature can then better capture the dependence distribution of LoS with the value of the categorical feature.

For the linear regression model [ 54 ], we sampled a set of 6 categorical features, [‘Type of Admission’, ‘Patient Disposition’, ‘APR Severity of Illness Code’, ‘APR Medical Surgical Description’, ‘APR MDC Code’] which we target encoded with the mean of the LoS and the median of the LoS. We then one-hot encoded every feature (all features are categorical) and for each such one-hot encoded feature, created a new feature for each of the features in the sampled set, by replacing the ones in the one-hot encoded feature with the value of the corresponding feature in the sampled set. For example, we one-hot encoded ‘Operating Certificate Number’, and for samples where ‘Operating Certificate Number’ was 3, we created 6 features, each where samples having the value 3 were assigned the target encoded values of the sampled set features, and the other samples were assigned zero. We used such techniques to exploit the linear relation between LoS and each feature.

According to the sklearn documentation [ 55 ], a random forest regressor is “a meta estimator that fits a number of decision tree regressors on various sub-samples of the dataset and uses averaging to improve the predictive accuracy and control over-fitting”. The random forest regressor leverages ensemble learning based on many randomized decision trees to make accurate and robust predictions for regression problems. The averaging of many trees protects against single trees overfitting the training data.

The random forest classifier is also an ensemble learning technique and uses many randomized decision trees to make predictions for classification problems. The 'wisdom of crowds' concept suggests that the decision made by a larger group of people is typically better than an individual. The random forest classifier uses this intuition, and allows each decision tree to make a prediction. Finally, the most popular predicted class is chosen as the overall classification.

For the Random Forest Regressor [ 56 , 57 ] and Random Forest Classifier [ 58 ], we only used a similar distribution dependent target encoding as a random forest classifier/ regressor is unsuitable for sparse one-hot encoded columns.

Multinomial logistic regression is a type of regression analysis that predicts the probabilities of the different possible outcomes of a categorically distributed dependent variable, given a set of independent variables. It allows for more than two discrete outcomes, extending binomial logistic regression for binary classification to models with multiple class membership. For the multinomial logistic regression model [ 59 ], we used only one-hot encoding, and not target encoding, as the target value was categorical.

Finally, we experimented with combinations of target encoding and one-hot encoding. We can either use target encoding, or one-hot encoding, or both. When both encodings are employed, the dimensionality of the data increases to accommodate the one-hot encoded features. For each combination of encodings, we also experimented with different regression models including linear regression and random forest regression.

Feature importance, selection, and feature engineering

We experimented with different feature selection methods. Since the focus of our work is on developing interpretable and explainable models, we used SHAP analysis to determine relevant features.

We examine the importance of different features in the dataset. We used the SHAP value (Shapley Additive Explanations), a popular measure for feature importance [ 60 ]. Intuitively, the SHAP value measures the difference in model predictions when a feature is used versus omitted. It is captured by the following formula.

where \({{\varnothing }}_{i}\) is the SHAP value of feature \(i\) , \(p\) is the prediction by the model, n is the number of features and S is any set of features that does not include the feature \(i\) . The specific model we used for the prediction was the random forest regressor where we target-encoded all features with the product of the mean and the median of the LoS, since most of the features were categorical.

Classification models

One approach to the problem is to bin the LoS into different classes, and train a classifier to predict which class an input sample falls in. We binned the LoS into roughly balanced classes as follows: 1 day, 2 days, 3 days, 4–6 days, > 6 days. This strategy is based on the distribution of the LoS as shown earlier in Figs.  3 and  4 .

A density plot of the distribution of the length of stay. The area under the curve is 1. We used a kernel density estimation with a Gaussian kernel [ 61 ] to generate the plot

We used three different classification models, comprising the following:

Multinomial Logistic Regression

Random Forest Classifier

CatBoost classifier [ 62 ].

We used a Multinomial Logistic Regression model [ 59 ] trained and tested using tenfold cross validation to classify the LoS into one of the bins. The multinomial logistic regression model is capable of providing explainable results, which is part of the requirements. We used the feature engineering techniques described in the previous section.

We used a Random Forest Classifier model trained and tested using tenfold cross validation to classify the LoS into one of the bins. We used a maximum depth of 10 so as to get explainable insights into the model.

Finally, we used a CatBoost Classifier model trained and tested using tenfold cross validation to classify the LoS into one of the bins.

Regression models

We used three different regression models with the feature engineering techniques mentioned above ( Feature encoding section). These comprise:

Linear regression

Catboost regression

Random forest regression

The linear regression was implemented using the nn.Linear() function in the open source library PyTorch [ 63 ]. We used the ‘Adam’ optimization algorithm [ 64 ] in mini-batch settings to train the model weights for linear regression.

We investigated CatBoost regression in order to create models with minimal feature sets, whereby models with a low number of input features would provide adequate results. Accordingly, we trained a CatBoost Regressor [ 65 ] in order to determine the relationship between combinations of features and the prediction accuracy as determined by the R 2 correlation score.

The random forest regression was implemented using the function RandomForestRegressor() in scikit learn [ 55 ].

Model performance measures

For the regression models, we used the following metrics to compare the model performance.

The R 2 score and the p -value. We use a significance level of α = 0.05 (5 %) for our statistical tests.  If the p -value is small, i.e. less than α = 0.05, then the R 2 score is statistically significant.

For classifier models, we used the following metrics to compare the model performance.

True positive rate, false negative rate, and F1 score [ 66 ].

We computed the Brier score using Brier’s original calculation in his paper [ 67 ]. In this formulation, for R classes the Brier score B can vary between 0 and R, with 0 being the best score possible.

where \({\widehat{y}}_{i,c}\) is the class probability as per the model and \({I}_{i,c}=1\) if the i th sample belongs to class c and \({I}_{i,c}=0\) if it does not belong to class c .

We used the Delong test [ 68 ] to compare the AUC for different classifiers.

These metrics will allow other researchers to replicate our study and provide benchmarks for future improvements.

In this section we present the results of applying the techniques in the Methods section.

Descriptive statistics

We provide descriptive statistics that help the reader understand the distributions of the variables of interest.

Table 1 summarizes basic statistical properties of the LoS variable.

Figure  5 shows the distribution of the LoS variable for newborns.

This figure depicts the distribution of the LoS variable for newborns

Table 2 shows the top 20 APR DRG descriptions based on their frequency of occurrence in the dataset.

Figure  6 shows the distribution of the LoS variable for the top 20 most frequently occurring APR DRG descriptions shown in Table  2 .

A 3-d plot showing the distribution of the LoS for the top-20 most frequently occuring APR DRG descriptions. The x-axis (horizontal) depicts the LoS, the y-axis shows the APR DRG codes and the z-axis shows the density or frequency of occurrence of the LoS

We experimented with different encoding schemes for the categorical variables and for each encoding we examined different regression techniques. Our results are shown in Table 3 . We experimented with the three encoding schemes shown in the first column. The last row in the table shows a combination of one-hot encoding and target encoding, where the number of columns in the dataset are increased to accommodate one-hot encoded feature values for categorical variables.

Feature importance, selection and feature engineering

We obtained the SHAP plots using a Random Forest Regressor trained with target-encoded features.

Figures  7  and 8 show the SHAP values plots obtained for the features in the newborn partition of the dataset. We find that the features, “APR DRG Code”, “APR Severity of Illness Code”, “Patient Disposition”, “CCS Procedure Code”, are very useful in predicting the LoS. For instance, high feature values for “APR Severity of Illness Code”, which are encoded by red dots have higher SHAP values than the blue dots, which correspond to low feature values.

SHAP Value plot for newborns

1-D SHAP plot, in order of decreasing feature importance: top to bottom (for non-newborns)

A similar interpretation can be applied to the features in the non-newborn partition of the dataset. We note that “Operating Certificate Number” is among the top-10 most important features in both the newborn and non-newborn partitions. This finding is discussed in the Discussion section.

From Fig.  9 , we observe that as the severity of illness code increases from 1–4, there is a corresponding increase in the SHAP values.

A 2-D plot showing the relationship between SHAP values for one feature, “APR Severity of Illness Code”, and the feature values themselves (non-newborns)

To further understand the relationship between the APR Severity of Illness code and the LoS, we created the plot in Fig.  10 . This shows that the most frequently occurring APR Severity of Illness code is 1 (Minor), and that the most frequently occurring LoS is 2 days. We provide this 2-D projection of the overall distribution of the multi-dimensional data as a way of understanding the relationship between the input features and the target variable, LoS.

A density plot showing the relationship between APR Severity of Illness Code and the LoS. The color scale on the right determines the interpretation of colors in the plot. We used a kernel density estimation with a Gaussian kernel [ 61 ] to generate the plot

Similarly, Fig.  11 shows the relationship between the birth weight and the length of stay. The most common length of stay is two days.

A density plot showing the distribution of the birth weight values (in grams) versus the LoS. The colorbar on the right shows the interpretation of color values shown in the plot. We used a kernel density estimation with a Gaussian kernel [ 61 ] to generate the plot

Classification

We obtained a classification accuracy of 46.98% using Multinomial Logistic Regression with tenfold cross-validation in the 5-class classification task for non-newborn cases. The confusion matrix in Fig.  12 shows that the highest density of correctly classified samples is in or close to the diagonal region. The regions where out model fails occurs between adjacent classes as can be inferred from the given confusion matrix.

Confusion matrix for classification of non-newborns. The number inside each square along the diagonal represents the number of correctly classified samples. The color is coded so lighter colors represent lower numbers

For the newborn cases, we obtained a classification accuracy of 60.08% using Random Forest Classification model with tenfold cross-validation in the 5-class classification task. The confusion matrix in Fig.  13 shows that the majority of data samples lie in or close to the diagonal region. The regions where our model does not do well occurs between adjacent classes as can be inferred from the given confusion matrix,

Confusion matrix for classification of newborns. The number inside each square along the diagonal represents the number of correctly classified samples. The color is coded so lighter colors represent lower numbers

The density plot in Fig.  14 shows the relationship between the actual LoS and the predicted LoS. For a LoS of 2 days, the centroid of the predicted LoS cluster is between 2 and 3 days.

Shows the density plot of the predicted length of stay versus actual length of stay for the classifier model for non-newborns. We used a kernel density estimation with a Gaussian kernel [ 61 ] to generate the plot

A quantitative depiction of our model errors is shown in Fig.  15 . The values in Fig.  15 are interpreted as follows. Referring to the column for LoS = 2, the top row shows that 51% of the predicted LoS values for an actual stay of 2 days is also 2 days (zero error), and that 23% of the predicted values for LoS equal to 2 days have an error of 1 day and so on. The relatively high values in the top row indicates that the model is performing well, with an error of less than 1 day. There are relatively few instances of errors between 2 and 3 days (typically less than 10% of the values show up in this row). The only exception is for the class corresponding to LoS great than 8 days. The truncation of the data to produce this class results in larger model errors specifically for this class.

Shows the distribution of correctly predicted LoS values for each class used in our model. Along the columns, we depict the different classes used in the model, consisting of LoS equal to 1, 2, 3 …8, and more than 8. Each row depicts different errors made in the prediction. For instance, the top row depicts an error of less than or equal to one day between the actual LoS and the predicted Los. The second row from the top depicts an error which is greater than 1 and less than or equal 2 days. And so on for the other rows, for non-newborns

Figures  16 and 17 show the scatter plots for the linear regression models. The exact line represents a line with slope 1, and a perfect model would be one that produced all points lying on this line.

Scatter plot showing an instance of a linear regression fit to the data (newborns). The R 2 score is 0.82. The blue line represents an exact fit, where the predicted LoS equals the actual LoS (slope of the line is 1)

Scatter plot for linear regression. (non-newborns). The R 2 score is 0.42. The blue line represents an exact fit, where the predicted LoS equals the actual LoS (slope of the line is 1)

Figure  18 shows a density plot depicting the relationship between the predicted length of stay and the actual length of stay.

Shows the density plot of the predicted length of stay versus actual length of stay for the classifier model for non-newborns. We used a kernel density estimation with a Gaussian kernel [ 40 ] to generate the plot. The best fit regression line to our predictions is shown in green, whereas the blue line represents the ideal fit (line of slope 1, where actual LoS and predicted LoS are equal)

Most of the existing literature on LoS stay prediction is based on data for specific disease conditions such as cancer or cardiac disease. Hence, in order to understand which CCS diagnosis codes produce good model fits, we produced the plot in Fig.  19 .

This figure shows the three CCS diagnosis codes that produced the top three R 2 scores using linear regression. These are 101, 100 and 109. The three CCS Diagnosis codes that produced the lowest R 2 scores are 159, 657, and 659

We provide the following descriptions in Tables  4  and 5 for the 3 CCS Diagnosis Codes in Fig.  19 with the top R 2 Scores using linear regression.

Similarly, the following table shows the 3 CCS Diagnosis Codes in Fig.  19 for the lowest R 2 Scores using linear regression.

Models with minimal feature sets

We trained a CatBoost Regressor [ 65 ] on the complete dataset in order to determine the relationship between combinations of features and the prediction accuracy as determined by the R 2 correlation score. This is shown in Fig.  20

The labels for each row on the left show combinations of different input features. A CatBoost regression model was developed using the selected combination of features. The R 2 correlation scores for each model is shown in the bar graph

We can infer from Fig.  20 that only four features (‘'APR MDC Code', 'APR Severity of Illness Code', 'APR DRG Code', 'Patient Disposition') are sufficient for the model to reach very close to its maximum performance. We obtain similar concurring results when using other regression models for the same experiment.

Classification trees

We used a random forest tree approach to generate the trees in Figs.  21 and 22 .

A random forest tree that represents a best-fit model to the data for newborns. With 4 levels of the decision tree, the R 2 score is 0.65

A random forest tree using only a tree of depth 3 that represents a best-fit model to the data for non-newborns. The R 2 score is 0.28. We can generate trees with greater depth that better fit the data, but we have shown only a depth of 3 for the sake of readability in the printed version of this paper. Otherwise, the tree would be too large to be legible on this page. The main point in this figure is to showcase the ease of interpretation of the working of the model through rules

We used tenfold cross validation to determine the regression scores. The results are summarized in Tables  6 and 7 .

We computed the multi-class classifier metrics for logistic regression, using one-hot encoding for non-newborns. The results are presented in Table  8 . The first row represents the accuracy of the classifier when Class 0 is compared against the rest of the classes. A similar interpretation applies to the other rows in the table, ie one-versus-rest. The macro average gives the balanced recall and precision, and the resulting F1 score. The weighted average gives a support (number of samples) weighted average of the individual class metric. The overall accuracy is computed by dividing the total number of accurate predictions, which is 49,686 out of a total number of 105,932 samples, which yields a value of 0.47.

For the category of non-newborns, Fig.  23  provides a graphical plot that visualizes the ROC curves for the different multiclass classifiers we developed.

This figure applies to data concerning non-newborns. We show the multiclass ROC curves for the performance of the catboost classifier for the different classes shown. The area under the ROC curve is 0.7844

In Table  9 we compare the performance of our multiclass classifier using logistic regression developed on 2016 SPARCS data against 2017 SPARCS data.

In order to compare the performance of the different classifiers, we computed the AUC measures reported in Table  10 . Figure 24 visualizes the data in Table 10 and Fig. 25 visualizes the data in Table 11 . In Tables 12 and 13 we report the results of computing the Delong test for non-newborns and newborns respectively. In Tables 14 and 15 we report the results of computing the Brier scores for non-new borns and newborns respectively.

A bar chart that depicts the data in Table  10 for non-newborns

A bar chart that depicts the data in Table  11

Model parameters

In Table  16 we present the parameter and hyperparameter values used in the different models.

Additional results shown in the Appendix/Supplementary material

Due to space restrictions, we show additional results in the Appendix/Supplementary Material. These results are in tabular form and describe the R 2 scores for different segmentations of the variables in the dataset, e.g. according to age group, severity of illness code, etc.

The most significant result we obtain is shown in Figs.  21 and 22 , which provides an interpretable working of the decision trees using random forest modeling. Figure  21 for newborns shows that the birth weight features prominently in the decision tree, occurring at the root node. Low birth weights are represented on the left side of the tree and are typically associated with longer hospital stays. Higher birth weights occur on the right side of the tree, and the node in the bottom row with 189,574 samples shows that the most frequently occurring predicted stay is 2.66 days. Figure  22 for non-newborns shows that the features of “APR DRG Code”, “APR Severity of Illness Code” and “Patient Disposition” are the most important top-level features to predict the LoS. This provides a relatively simple rule-based model, which can be easily interpreted by healthcare providers as well as patients. For instance, the right-most branch of the tree classifies the input data into a relatively high LoS (46 days) when the branch conditions APR DRG Code is greater than 813.55 and the APR Severity of Illness Code is less than 91.

The results in Fig.  19 and Table  4 show that if we restrict our model to specific CCS Diagnosis descriptions such as “coronary atherosclerosis and other heart disease”, we obtain a good R 2 Score of 0.62. The objective of our work is not to cherry-pick CCS Diagnosis codes that produce good results, but rather to develop a single model for the entire SPARCS dataset to obtain a birds-eye perspective. For future work, we can explicitly build separate models for each CCS Diagnosis code, and that could have relevance to specific medical specialties, such as cardiovascular care.

Similarly, the results in Fig.  19 and Table  5 show that there are CCS Diagnosis codes corresponding to schizophrenia and mood disorders that produce a poor model fit. Factors that contribute to this include the type of data in the SPARCS dataset, where information about patient vitals, medications, or a patient’s income level is not provided, and the inherent variability in treating schizophrenia and mood disorders. Baeza et al. [ 69 ] identified several variables that affect the LoS in psychiatric patients, which include psychiatric admissions in the previous years, psychiatric rating scale scores, history of attempted suicide, and not having sufficient income. Such variables are not provided in the SPARCS dataset. Hence a policy implication is to collect and make such data available, perhaps as a separate dataset focused on mental health issues, which have proven challenging to treat.

Figures  16 and 17 show that a better regression fit is obtained when a specific CCS Diagnosis code is used to build the model, such as “Newborn” in Fig.  16 . To put these results in context, we note that it is difficult to obtain a high R 2 value for healthcare datasets in general, and especially for large numbers of patient samples that span multiple hospitals. For instance, Bertsimas [ 70 ] reported an R 2 value of 0.2 and Kshirsagar [ 71 ] reported an R 2 value of 0.33 for similar types of prediction problems as studied in this paper.

Further details for a segmentation of R 2 scores by the different variable categories are shown in the Appendix/Supplementary Material section. For instance, the table corresponding to Age Groups shows that there is close agreement between the mean of the predicted LoS from our model and the actual LoS. Furthermore, the mean LoS increases steadily from 4.8 days for Age group 0–17 to 6.4 days for ages 70 or older. A discussion of these tables is outside the scope of this paper. However, they are being provided to help other researchers form hypotheses for further investigations or to find supporting evidence for ongoing research.

Table 3 shows that the best encoding scheme is to combine target encoding with one-hot encoding and then apply linear regression. This produces an R 2 score of 0.42 for the non-newborn data, which is the best fit we could obtain. This table also shows that significant improvements can be obtained by exploring the search space which consists of different strategies of feature encoding and regression methods. There is no theoretical framework which determines the optimum choice, and the best method is to conduct an experimental search. An important contribution of the current paper is to explore this search space so that other researchers can use and build upon our methodology.

The distribution of errors in Fig.  15 shows that the truncation we employed at a LoS of 8 days produces artifacts in the prediction model as all stays of greater than 8 days are lumped into one class. Nevertheless, the distribution of LoS values in Fig.  4 shows that a relatively small number of data samples have LoS greater than 8 days. In the future, we will investigate different truncation levels, and this is outside the scope of the current paper. By using our methodology, the truncation level can also be tuned by practitioners in the field, including hospital administrators and other researchers.

Our results in Fig.  7 show that certain features are not useful in predicting the LoS. The SHAP plot shows that features such as race, gender, and ethnicity are not useful in predicting the LoS. It would have been interesting if this were not the case, as that implies that there is systemic bias based on race, gender or ethnicity. For instance, a person with a given race may have a smaller LoS based on their demographic identity. This would be unacceptable in the medical field. It is satisfying to see that a large and detailed healthcare dataset does not show evidence of bias.

To place this finding in context, racial bias is an important area of research in the U.S., especially in fields such as criminology and access to financial services such as loans. In the U.S., it is well known that there is a disproportional imprisonment of black and Hispanic males [ 72 ]. Researchers working on criminal justice have determined that there is racial bias in the process of sentencing and granting parole, with blacks being adversely affected [ 73 ]. This bias is reinforced through any algorithms that are trained on the underlying data. There is evidence that banks discriminate against applicants for loans based on their race or gender [ 74 ].

This does not appear to be the case in our analysis of the SPARCS data. Though we did not specifically investigate the issue of racial bias in the LoS, the feature analysis we conducted automatically provides relevant answers. Other researchers including those in the U.K [ 21 ] have also determined that gender does not have an effect on LoS or costs. Hence the results in the current paper are consistent with the findings of other researchers in other countries working on entirely different datasets.

From Table  6 we see that in the case of data concerning non-newborns, the catboost regression performs the best, with an R 2 score of 0.432. The p -value is less than 0.01, indicating that the correlation between the actual and predicted values of LoS through catboost regression is statistically significant. Similarly, the p -values for linear regression and random forest regression indicate that these models produce predictions that are statistically significant, i.e. they did not occur by random chance.

From Table  7 that refers to data from newborns, the linear regression performs the best, with an R 2 score of 0.82. The p -value is less than 0.01, indicating that the correlation between the actual and predicted values of LoS through linear regression is statistically significant. Similarly, the p -values for random forest regression and catboost regression indicate that these models produce predictions that are statistically significant.

We examine the performance of classifiers on non-newborn data, as shown in Tables  10 and 12 . The Delong test conducted in Table  12 shows that there is a statistically significant difference between the AUCs of the pairwise comparisons of the models. Hence, we conclude that the catboost classifier performs the best with an average AUC of 0.7844. We also note that there is a marginal improvement in performance when we use the catboost classifier instead of the random forest classifier. Both the catboost classifier and the random forest classifier perform better than logistic regression. We conclude that the best performing model for non-newborns is the catboost classifier, followed by the random forest classifier, and then logistic regression.

In the case of newborn data, we examine the performance of the classifiers as shown in Tables  11 and 13 . From Table 13 , we note that the p -values in all the rows are less than 0.05, except for the binary class “one vs. rest for class 3”, random forests vs. catboost. Hence, for this particular comparison between the random forest classifier and the catboost classifier for “one vs. rest for class 3”, we cannot conclude that there is a statistically significant difference between the performance of these two classifiers. From Table  11 we observe that the AUCs of these two classifiers are very similar. We also note that only about 10% of the dataset consists of newborn cases.

From Table  14 we note that the Brier score for the catboost classifier is the lowest. A lower Brier score indicates better performance. According to the Brier scores for the non-newborn data, the catboost classifier performs the best, followed by the random forest classifier and then logistic regression. Table 15 shows that for newborns, the random forest classifier performs the best, followed by the catboost classifier and logistic regression. The performance of the random forest classifier and catboost classifier are very similar.

From a practical perspective, it may make sense to use a catboost classifier on both newborn and non-newborn data as it simplifies the processing pipeline. The ultimate decision rests with the administrators and implementers of these decision systems in the hospital environment.

Burn et al. observe [ 21 ] that though the U.S. has reported similar declines in LoS as in the U.K, the overall costs of joint replacement have risen. The U.K. government created policies to encourage the formation of specialist centers for joint replacement, which have resulted in reduction in the LoS as well as delivering cost reductions. The results and analysis presented in our current paper can help educate patients and healthcare consumers about trends in healthcare costs and how they can be reduced. An informed and educated electorate can press their elected representatives to make changes to the healthcare system to benefit the populace.

Hachesu et al. examined the LoS for cardiac disease patients [ 22 ] where they used data from around 5000 patients and considered 35 input variables to build a predictive model. They found that the LoS was longer in patients with high blood pressure. In contrast, our method uses data from 2.5 million patients and considers multiple disease conditions simultaneously. We also do not have access to patient vitals such as blood pressure measurements, due to the limitation of the existing New York State SPARCS data.

Garcia et al. [ 23 ] conducted a study of elderly patients (age greater than 60) to understand factors governing the LoS for hip fracture treatment. They used 660 patient records and determined that the most significant variable was the American Society of Anesthesiologists (ASA) classification system. The ASA score ranges from 1–5 and captures the anesthesiologist’s impression of a patient’s health and comorbidities at the time of surgery. Garcia et al. showed a monotonically increasing relationship between the ASA score and the LoS. However, they did not build a specific predictive model. Their work shows that it is possible to find single variables with significant information content in order to estimate the LoS. The New York SPARCS dataset that we used does not contain the ASA score. Hence a policy implication of our research is to alert the healthcare authorities include such variables such as the ASA score where relevant in the datasets released in the future. The additional storage required is very small (one additional byte per patient record).

Arjannikov et al. [ 25 ] developed predictive models by binarizing the data into two categories, e.g. LoS <  = 2 days or LoS > 2 days. In our work, we did not employ such a discretization. In contrast, we used continuous regression techniques as well as classification into more than two bins. It is preferable to stay as close to the actual data as possible.

Almashrafi et al. [ 27 ] and Cots et al. [ 75 ] observed that larger hospitals tended to have longer LoS for patients undergoing cardiac surgery. Though we did not specifically examine cardiac surgery outcomes, our feature analysis indicated that the hospital operating certificate number had lower relevance than other features such as DRG codes. Nevertheless, the SHAP plots in Fig.  7 and Fig.  8 show that the hospital operating certificate number occurs within the top 10 features in order of SHAP values. We will investigate this relationship in more detail in future research, as it requires determining the size of the hospital from the operating certificate number and creating an appropriate machine-learning model. The Appendix contains results that show certain operating certificate numbers that produce a good model fit to the data.

A major focus of our research is on building interpretable and explainable models. Based on the principle of parsimony, it is preferable to utilize models which involve fewer features. This will provide simpler explanations to healthcare professionals as well as patients. We have shown through Fig.  20 that a model with five features performs just as well as a model with seven features. These features also make intuitive sense and the model’s operation can be understood by both patients and healthcare providers.

Patients in the U.S. increasingly have to pay for medical procedures out-of-pocket as insurance payments do not cover all the expenses, leading to unexpectedly large bills [ 76 ]. Many patients also do not possess health insurance in the U.S., with the consequence that they get charged the highest [ 77 ]. Kullgreen et.al. observe that patients in the U.S. need to be discerning healthcare consumers [ 78 ], as they can optimize the value they receive from out-of-pocket spending. In addition to estimating the cost of medical procedures, patients will also benefit from estimating the expected duration for a procedure such as joint replacement. This will allow them to budget adequate time for their medical procedures. Patients and consumers will benefit from obtaining estimates from an unbiased open data source such as New York State SPARCS and the use of our model.

Other researchers have developed specific LoS models for particular health conditions, such as cardiac disease [ 22 ], hip replacement [ 21 ], cancer [ 26 ], or COVID-19 [ 24 ]. In addition, researchers typically assume a prior statistical distribution for the outcomes, such a Weibull distribution [ 24 ]. However, we have not made any assumptions of specific prior statistical distributions, nor have we restricted our analysis to specific diseases. Consequently, our model and techniques should be more widely applicable, especially in the face of rapidly changing disease trajectories worldwide.

Our study is based exclusively on freely available open health data. Consequently, we cannot control the granularity of the data and must use the data as-is. We are unable to obtain more detailed patient information such as their physiological variables such as blood pressure, heartrate variability etc. at the time of admittance and during their stay. Hospitals, healthcare providers, and insurers have access to this data. However, there is no mandate for them to make this available to researchers outside their own organizations. Sometimes they sell de-identified data to interested parties such as pharmaceutical companies [ 79 ]. Due to the high costs involved in purchasing this data, researchers worldwide, especially in developing countries are at a disadvantage in developing AI algorithms for healthcare.

There is growing recognition that medical researchers need to standardize data formats and tools used for their analysis, and share them openly. One such effort is the organization for Observational Health Data Sciences and Informatics (OHDSI) as described in [ 80 ].

Twitter has demonstrated an interesting path forward, where a small percentage of its data was made available freely to all users for non-commercial purposes through an API [ 81 ]. Recently, Twitter has made a larger proportion of its data available to qualified academic researchers [ 82 ]. In the future, the profit motives of companies need to be balanced with considerations for the greater public good. An advantage of using the Twitter model is that it spurs more academic research and allows universities to train students and the workforce of the future on real-world and relevant datasets.

In the U.S., a new law went into effect in January 2021 requiring hospitals to make pricing data available publicly. The premise is that having this data would provide better transparency into the working of the healthcare system in the U.S. and lead to cost efficiencies. However, most hospitals are not in compliance with this law [ 83 ]. Concerted efforts by government officials as well as pressure by the public will be necessary to achieve compliance. If the eventual release of such data is not accompanied by a corresponding interest shown by academicians, healthcare researchers, policymakers, and the public it is likely that the very premise of the utility of this data will be called into question. Furthermore, merely dumping large quantities of data into the public domain is unlikely to benefit anyone. Hence research efforts such as the one presented in this paper will be valuable in demonstrating the utility of this data to all stakeholders.

Our machine-learning pipeline can easily be applied to new data that will be released periodically by New York SPARCS, and also to hospital pricing data [ 83 ]. Due to our open-source methodology, other researchers can easily extend our work and apply it to extract meaning from open health data. This improves reproducibility, which is an essential aspect of science. We will make our code available on Github to interested researchers for non-commercial purposes.

Limitations of our models

Our models are restricted to the data available through New York State SPARCS, which does not provide detailed information about patient vitals. More detailed physiological data is available through the Multiparameter Intelligent Monitoring in Intensive Care (MIMIC) framework [ 84 ], though for a smaller number of patients. We plan to extend our methodology to handle such data in the future. Another limitation of our study is that it does not account for patient co-morbidities. This arises from the de-identification process used to release the SPARCS data, where patient information is removed. Hence we are unable to analyze multiple hospital admissions for a given patient, possibly for different conditions. The main advantage of our approach is that it uses large-scale population data (2.3 million patients) but at a coarse level of granularity, where physiological data is not available. Nevertheless, our approach provides a high-level view of the operation of the healthcare system, which provides valuable insights.

There is growing interest in using data analytics to increase government transparency and inform policymaking. It is expected that the meaning and insights gained from such evidence-based analysis will translate to better policies and optimal usage of the available infrastructure. This requires cooperation between computer scientists, domain experts, and policy makers. Open healthcare data is especially valuable in this context due to its economic significance. This paper presents an open-source analytics system to conduct evidence-based analysis on openly available healthcare data.

The goal is to develop interpretable machine learning models that identify key drivers and make accurate predictions related to healthcare costs and utilization. Such models can provide actionable insights to guide healthcare administrators and policy makers. A specific illustration is provided via a robust machine learning pipeline that predicts hospital length of stay across 285 disease categories based on 2.3 million de-identified patient records. The length of stay is directly related to costs.

We focused on the interpretability and explainability of input features and the resulting models. Hence, we developed separate models for newborns and non-newborns, given differences in input features. The best performing model for non-newborn data was catboost regression, which used linear regression and achieved an R 2 score of 0.43. The best performing model for newborns and non-newborns respectively was linear regression, which achieved an R 2 score of 0.82. Key newborn predictors included birth weight, while non-newborn models relied heavily on the diagnostic related group classification. This demonstrates model interpretability, which is important for adoption. There is an opportunity to further improve performance for specific diseases. If we restrict our analysis to cardiovascular disease, we obtain an improved R 2 score of 0.62.

The presented approach has several desirable qualities. Firstly, transparency and reproducibility are enabled through the open-source methodology. Secondly, the model generalizability facilitates insights across numerous disease states. Thirdly, the technical framework can easily integrate new data while allowing modular extensions by the research community. Lastly, the evidence generated can readily inform multiple key stakeholders including healthcare administrators planning capacity, policy makers optimizing delivery, and patients making medical decisions.

Availability of data and materials

Data is publicly available at the website mentioned in the paper, https://www.health.ny.gov/statistics/sparcs/

There is an “About Us” tab in the website which contains all the contact details. The authors have nothing to do with this website as it is maintained by New York State.

Gurría A. Openness and Transparency - Pillars for Democracy, Trust and Progress. OECD.org. Available: https://www.oecd.org/unitedstates/opennessandtransparency-pillarsfordemocracytrustandprogress.htm . Accessed 28 June 2024.

Jetzek T. The Sustainable Value of Open Government Data: Uncovering the Generative Mechanisms of Open Data through a Mixed Methods Approach. lCopenhagen Business School, Institut for IT-Ledelse Department of IT Management. 2015.

Move fast and heal things: How health care is turning into a consumer product. The Economist. 2022.  https://www.economist.com/business/how-health-care-is-turning-into-a-consumer-product/21807114 . Accessed 28 June 2024.

New York State Department Of Health, Statewide Planning and Research Cooperative System (SPARCS).  https://www.health.ny.gov/statistics/sparcs/ . Accessed 5 Oct 2022.

Rao AR, Chhabra A, Das R, Ruhil V. A framework for analyzing publicly available healthcare data. In 2015 17th International Conference on E-health Networking, Application & Services (IEEE HealthCom). 2015: IEEE, pp. 653–656.

Rao AR, Clarke D. A fully integrated open-source toolkit for mining healthcare big-data: architecture and applications. In IEEE International Conference on Healthcare Informatics ICHI, Chicago. 2016: IEEE, pp. 255–261.

Rao AR, Garai S, Dey S, Peng H. PIKS: A Technique to Identify Actionable Trends for Policy-Makers Through Open Healthcare Data. SN Computer Science. 2021;2(6):1–22.

Article   Google Scholar  

Rao AR, Rao S, Chhabra R. Rising mental health incidence among adolescents in Westchester, NY. Community Ment Health J. 2021:1–1. 

Boylan J F. My $145,000 Surprise Medical Bill. New York Times. 2020.  https://www.nytimes.com/2020/02/19/opinion/surprise-medical-bill.html . Accessed 28 June 2024.

Peterson K, Bykowicz J. Congress Debates Push to End Surprise Medical Billing. Wall Street J. 2020.  https://www.wsj.com/articles/congress-debates-push-to-end-surprise-medical-billing-11589448603 . Accessed 28 June 2024.

Wang S, Zhang J, Fu Y, Li Y. ACM TIST Special Issue on Deep Learning for Spatio-Temporal Data: Part 1. 12th ed. NY: ACM New York; 2021. p. 1–3.

Google Scholar  

Jones R. lining length of stay and future bed numbers. BJHCM. 2015;21(9):440–1.

Daghistani TA, Elshawi R, Sakr S, Ahmed AM, Al-Thwayee A, Al-Mallah MH. Predictors of in-hospital length of stay among cardiac patients: a machine learning approach. Int J Cardiol. 2019;288:140–7.

Article   PubMed   Google Scholar  

Sen-Crowe B, Sutherland M, McKenney M, Elkbuli A. A closer look into global hospital beds capacity and resource shortages during the COVID-19 pandemic. J Surg Res. 2021;260:56–63.

Article   CAS   PubMed   Google Scholar  

Stone K, Zwiggelaar R, Jones P, Mac Parthaláin N. A systematic review of the prediction of hospital length of stay: Towards a unified framework. PLOS Digital Health. 2022;1(4):e0000017.

Article   PubMed   PubMed Central   Google Scholar  

Lequertier V, Wang T, Fondrevelle J, Augusto V, Duclos A. Hospital length of stay prediction methods: a systematic review. Med Care. 2021;59(10):929–38.

Sridhar S, Whitaker B, Mouat-Hunter A, McCrory B. Predicting Length of Stay using machine learning for total joint replacements performed at a rural community hospital. PLoS ONE. 2022;17(11);e0277479.

Article   CAS   PubMed   PubMed Central   Google Scholar  

CCS (Clinical Classifications Software) - Synopsis. https://www.nlm.nih.gov/research/umls/sourcereleasedocs/current/CCS/index.html . Accessed 13 Jan 2022.

Sotoodeh M, Ho JC. Improving length of stay prediction using a hidden Markov model. AMIA Summits on Translational Science Proceedings. 2019;2019:425.

PubMed Central   Google Scholar  

Ma F, Yu L, Ye L, Yao DD, Zhuang W. Length-of-stay prediction for pediatric patients with respiratory diseases using decision tree methods. IEEE J Biomed Health Inform. 2020;24(9):2651–62.

Burn E, et al. Trends and determinants of length of stay and hospital reimbursement following knee and hip replacement: evidence from linked primary care and NHS hospital records from 1997 to 2014. BMJ Open. 2018;8(1);e019146.

Hachesu PR, Ahmadi M, Alizadeh S, Sadoughi F. Use of data mining techniques to determine and predict length of stay of cardiac patients. Healthcare informatics research. 2013;19(2):121–9.

Garcia AE, et al. Patient variables which may predict length of stay and hospital costs in elderly patients with hip fracture. J Orthop Trauma. 2012;26(11):620–3.

Vekaria B, et al. Hospital length of stay for COVID-19 patients: Data-driven methods for forward planning. BMC Infect Dis. 2021;21(1):1–15.

Arjannikov T, Tzanetakis G. An empirical investigation of PU learning for predicting length of stay. In 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI). 2021: IEEE, pp. 41–47.

Gupta D, Vashi PG, Lammersfeld CA, Braun DP. Role of nutritional status in predicting the length of stay in cancer: a systematic review of the epidemiological literature. Ann Nutr Metab. 2011;59(2–4):96–106.

Almashrafi A, Elmontsri M, Aylin P. Systematic review of factors influencing length of stay in ICU after adult cardiac surgery. BMC Health Serv Res. 2016;16(1):318.

Kalgotra P, Sharda R. When will I get out of the hospital? Modeling Length of Stay using Comorbidity Networks. J Manag Inf Syst. 2021;38(4):1150–84.

Awad A, Bader-El-Den M, McNicholas J. Patient length of stay and mortality prediction: a survey. Health Serv Manage Res. 2017;30(2):105–20.

Editorial-Board. The Lancet, HCL and Trump. Wall Street J. 2020.  https://www.wsj.com/articles/the-lancet-hcl-and-trump-11591226880 . Accessed 28 June 2024.

Servick  K, Enserink M. A mysterious company’s coronavirus papers in top medical journals may be unraveling. Science. 2020.  https://www.science.org/content/article/mysterious-company-s-coronavirus-papers-top-medical-journals-may-be-unraveling . Accessed 28 June 2024.

Gabler E, Rabin RC. The Doctor Behind the Disputed Covid Data. New York Times. 2020.  https://www.nytimes.com/2020/07/27/science/coronavirus-retracted-studies-data.html . Accessed 28 June 2024.

Lancet-Editors. Expression of concern: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. 2020;395:10240. https://www.science.org/content/article/mysterious-company-s-coronavirus-papers-topmedical-journals-may-be-unraveling . Accessed 28 June 2024.

Editorial-Board. Expression of Concern: Mehra MR et al. Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19. N Engl J Med. 2020.  https://www.nejm.org/doi/full/10.1056/NEJMoa2007621 . Accessed 28 June 2024.

Hopkins JS, Gold R. Authors Retract Studies That Found Risks of Using Antimalaria Drugs Against Covid-19. Wall Street J. 2020. https://www.wsj.com/articles/authors-retract-study-that-found-risks-of-using-antimalaria-drug-against-covid-19-11591299329 . Accessed 28 June 2024.

https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(20)31180-6.pdf . Accessed 9 Jan 2022.

Wolfensberger M, Wrigley A. Trust in Medicine. Cambridge University Press. 2019. ISBN-13: 978-1108487191.

Bhattacharya J, Nicholson T. A Deceptive Covid Study, Unmasked. Wall Street J. 2022. https://www.wsj.com/articles/deceptive-covid-study-unmasked-abc-misleading-omicron-north-carolina-students-duke-mask-test-to-stay-11641933613 . Accessed 28 June 2024.

Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 2016;533(7604):452–4.

Begley CG, Ioannidis JP. Reproducibility in science: improving the standard for basic and preclinical research. Circ Res. 2015;116(1):116–26.

Eisner D. Reproducibility of science: Fraud, impact factors and carelessness. J Mol Cell Cardiol. 2018;114:364–8.

Wang F, Kaushal R, Khullar D. Should health care demand interpretable artificial intelligence or accept “black box” medicine? Am College Phys. 2020;172:59–60.

Reyes M, et al. On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiol Art Intell. 2020;2(3):e190043.

Savadjiev P, et al. Demystification of AI-driven medical image interpretation: past, present and future. Eur Radiol. 2019;29(3):1616–24.

McKinney W. Python for data analysis: Data wrangling with Pandas, NumPy, and IPython. " O’Reilly Media, Inc. 2012.

Pedregosa F, et al. Scikit-learn: Machine learning in Python. J Machine Learn Res. 2011;12:2825–30.

Cass S. The top programming languages: Our latest rankings put Python on top-again-[Careers]. IEEE Spectr. 2020;57(8):22–22.

Tjoa E, Guan C. A survey on explainable artificial intelligence (xai): Toward medical xai," IEEE Transactions on Neural Networks and Learning Systems. 2020.

https://www.health.ny.gov/statistics/sparcs/docs/sparcs_data_dictionary.xlsx . Accessed 28 June 2024.

Design and development of the Diagnosis Related Group (DRG). https://www.cms.gov/icd10m/version37-fullcode-cms/fullcode_cms/Design_and_development_of_the_Diagnosis_Related_Group_(DRGs).pdf . Accessed 5 Oct 2022.

ARTICLE 28, Hospitals, Public Health (PBH) CHAPTER 45. 2023. Available: https://www.nysenate.gov/legislation/laws/PBH/A28 . Accessed 28 June 2024.

Gilmore‐Bykovskyi A, et al. Disparities in 30‐day readmission rates among Medicare enrollees with dementia. J Am Geriatr Soc. 2023.

Rodríguez P, Bautista MA, Gonzalez J, Escalera S. Beyond one-hot encoding: Lower dimensional target embedding. Image Vis Comput. 2018;75:21–31.

Montgomery DC, Peck EA, Vining GG. Introduction to linear regression analysis. 6th ed. John Wiley & Sons; 2021. ISBN-13 978-1119578727.

Random forest regressor in sklearn. Available: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html . Accessed 28 June 2024.

Breiman L. Random forests. Mach Learn. 2001;45:5–32.

Svetnik V, Liaw A, Tong C, Culberson JC, Sheridan RP, Feuston BP. Random forest: a classification and regression tool for compound classification and QSAR modeling. J Chem Inf Comput Sci. 2003;43(6):1947–58.

Liaw A, Wiener M. Classification and regression by randomForest. R news. 2002;2(3):18–22.

Böhning D. Multinomial logistic regression algorithm. Ann Inst Stat Math. 1992;44(1):197–200.

Vaid A, et al. Machine Learning to Predict Mortality and Critical Events in a Cohort of Patients With COVID-19 in New York City: Model Development and Validation. J Med Internet Res. 2020;22(11);e24018.

Density Estimation.  https://scikit-learn.org/stable/modules/density.html . Accessed 5 Oct 2022.

CatBoost, a high-performance open source library for gradient boosting on decision trees. Available:  https://catboost.ai/  and https://catboost.ai/en/docs/concepts/python-usages-examples . Accessed 28 June 2024.

PyTorch documentation for torch.nn, the basic building blocks for graphs. Available: https://pytorch.org/docs/stable/nn.html . Accessed 28 June 2024.

Kingma DP, Ba J. Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980. 2014.

Prokhorenkova L, Gusev G, Vorobev A, Dorogush AV, Gulin A. CatBoost: unbiased boosting with categorical features," arXiv preprint arXiv:1706.09516. 2017.

Tharwat A. Classification assessment methods. Applied computing and informatics. 2020;17(1):168–92.

Brier GW. Verification of forecasts expressed in terms of probability. Mon Weather Rev. 1950;78(1):1–3.

DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988:837–45.

Baeza FL, da Rocha NS, Fleck MP. Predictors of length of stay in an acute psychiatric inpatient facility in a general hospital: a prospective study. Brazilian Journal of Psychiatry. 2017;40:89–96.

Bertsimas D, et al. Algorithmic prediction of health-care costs. Oper Res. 2008;56(6):1382–92.

Kshirsagar R. Accurate and Interpretable Machine Learning for Transparent Pricing of Health Insurance Plans," presented at the AAAI 2021 Conference. 2021.

Ulmer J, Painter-Davis N, Tinik L. Disproportional imprisonment of Black and Hispanic males: Sentencing discretion, processing outcomes, and policy structures. Justice Q. 2016;33(4):642–81.

Angwin J, J. Larso J, Mattu S, Kirchner L. Machine bias: There’s software used across the country to predict future criminals. And it’s biased against blacks. ProPublica (2016). Google Scholar. 2016;23.

Steil JP, Albright L, Rugh JS, Massey DS. The social structure of mortgage discrimination. Hous Stud. 2018;33(5):759–76.

Cots F, Mercadé L, Castells X, Salvador X. Relationship between hospital structural level and length of stay outliers: Implications for hospital payment systems. Health Policy. 2004;68(2):159–68.

Evans M, McGinty T. Hospital Prices Are Arbitrary. Just Look at the Kingsburys’ $100,000 Bill. Wall Street J. 2021.  https://www.wsj.com/articles/hospital-prices-arbitrary-healthcare-medical-bills-insurance-11635428943 . Accessed 28 June 2024.

Evans M. Hospitals Often Charge Uninsured People the Highest Prices, New Data Show. Wall Street J. 2021. https://www.wsj.com/articles/hospitals-often-charge-uninsured-people-the-highest-prices-new-data-show-11625584448 . Accessed 28 June 2024.

Kullgren JT, et al. A survey of Americans with high-deductible health plans identifies opportunities to enhance consumer behaviors. Health Aff. 2019;38(3):416–24.

Wetsman N. Hospitals are selling treasure troves of medical data — what could go wrong? The Verge. 2021. Available: https://www.theverge.com/2021/6/23/22547397/medical-records-health-data-hospitals-research . Accessed 28 June 2024.

Hripcsak G, et al. Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers. Stud Health Technol Inform. 2015;216:574–8.

PubMed   PubMed Central   Google Scholar  

Gabarron E, Dorronzoro E, Rivera-Romero O, Wynn R. Diabetes on Twitter: a sentiment analysis. J Diabetes Sci Technol. 2019;13(3):439–44.

Statt N. Twitter is opening up its full tweet archive to academic researchers for free. The Verge. 2021. Available: https://www.theverge.com/2021/1/26/22250203/twitter-academic-research-public-tweet-archive-free-access . Accessed 28 June 2024. 

Evans M, Mathews AW, McGinty T. Hospitals Still Not Fully Complying With Federal Price-Disclosure Rules. Wall Street J. 2021.  https://www.wsj.com/articles/hospital-price-public-biden-11640882507 .

Johnson AE, et al. MIMIC-III, a freely accessible critical care database. Scientific data. 2016;3(1):1–9.

Download references

Acknowledgements

We are grateful to the New York State SPARCS program for making the data available freely to the public. We greatly appreciate the feedback provided by the anonymous reviewers which helped in improving the quality of this manuscript.

No external funding was available for this research.

Author information

Authors and affiliations.

Indian Institute of Technology, Delhi, India

Raunak Jain, Mrityunjai Singh & Rahul Garg

Fairleigh Dickinson University, Teaneck, NJ, USA

A. Ravishankar Rao

You can also search for this author in PubMed   Google Scholar

Contributions

Raunak Jain, Mrityunjai Singh, A. Ravishankar Rao, and Rahul Garg contributed equally to all stages of preparation of the manuscript.

Corresponding author

Correspondence to A. Ravishankar Rao .

Ethics declarations

Ethics approval and consent to participate.

Not applicable as no human subjects were used in our study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Jain, R., Singh, M., Rao, A.R. et al. Predicting hospital length of stay using machine learning on a large open health dataset. BMC Health Serv Res 24 , 860 (2024). https://doi.org/10.1186/s12913-024-11238-y

Download citation

Received : 19 June 2023

Accepted : 24 June 2024

Published : 29 July 2024

DOI : https://doi.org/10.1186/s12913-024-11238-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Machine learning
• Artificial intelligence
• Health informatics
• Open-source software
• Healthcare analytics

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Venue: Shaw Centre, Ottawa
• ICPC 2025 Committees
• Organizing Committee
• Steering Committee
• Track Committees
• Program Committee
• Contributors
• People Index

Research Track ICPC 2025

Call for papers.

The 33rd EEE/ACM International Conference on Program Comprehension (ICPC) is the premier venue for work in the area of software program comprehension. It encompasses human activities for comprehending the software as well as methodologies and technologies for supporting such comprehension.

The research track provides a quality forum for researchers and practitioners from academia, industry, and government to present and discuss new results in program comprehension. Topics of interest for all tracks include but are not limited to:

• Empirical evaluations of program comprehension tools, techniques, and approaches;
• Human aspects in program comprehension, including collaborative software engineering practices, gender considerations, information processing strategies, the role of emotions, emotional awareness, and more;
• Cognitive theories for program comprehension accompanied by different empirical strategies, including experiments, surveys, and case studies;
• Topics at the intersection between program comprehension and programming education;
• Individual, collaborative, distributed, and global program comprehension;
• Novel visualization or summarization techniques and interfaces to support program comprehension, including searching, browsing, and analyzing;
• Comprehension of specific types of software systems, such as open/closed source, mobile applications, spreadsheets, web-based systems, legacy systems, product lines, libraries, multi-threaded applications, and systems of systems;
• Comprehension in the context of diverse software process models and specific lifecycle activities, such as: maintenance, evolution, re-engineering, migration, security, auditing, and testing;
• Comprehension of software artifacts ranging from requirements documents to test cases and crash logs; from API documentation to models, meta-models and model transformation; and from Stack Overflow questions & answers to GitHub code review messages or video tutorials - all software artifacts and formal or informal documentation that software developers encounter when creating or evolving software;
• Comprehension and legal issues, such as due diligence, intellectual property and licensing, reverse engineering, and litigation; Issues and case studies in the transfer of program comprehension technology to industry;
• Automated tool support for program comprehension.

A subset of the Research Track papers accepted for presentation at ICPC 2025 will be invited to be revised and extended for consideration in a thematic special issue of the Springer Empirical Software Engineering Journal (EMSE) . The best papers of the research track will be awarded with an ACM SIGSOFT Distinguished Paper Award at ICPC. In addition, ICPC 2025 will feature ICPC Honorable Mentions, special awards that will be assigned, on the basis of the program committee reports, to the papers that have applied extremely novel and/or outstanding research methods to the problem of interest.

Format and Submission

Submissions must conform to the IEEE conference proceedings template, specified in the IEEE Conference Proceedings Formatting Guidelines (title in 24pt font and full text in 10pt type, LaTeX users must use \documentclass[10pt,conference]{IEEEtran} without including the compsoc or compsocconf options).

All submissions must not exceed 10 pages for the main text, inclusive of all figures, tables, appendices, etc. Two more pages containing only references are permitted. All submissions must be in PDF. Accepted papers will be allowed one extra page for the main text of the camera-ready version.

• Submissions must strictly conform to the IEEE conference proceedings formatting instructions specified above. Alterations of spacing, font size, and other changes that deviate from the instructions may result in desk rejection without further review.
• By submitting to the ICPC Research Track, authors acknowledge that they are aware of and agree to be bound by the ACM Policy and Procedures on Plagiarism and the IEEE Plagiarism FAQ . Papers submitted to ICPC 2025 must not have been published elsewhere and must not be under review or submitted for review elsewhere whilst under consideration for ICPC 2025. Contravention of this concurrent submission policy will be deemed a serious breach of scientific ethics, and appropriate action will be taken in all such cases. To check for double submission and plagiarism issues, the chairs reserve the right to (1) share the list of submissions with the PC Chairs of other conferences with overlapping review periods and (2) use external plagiarism detection software, under contract to the ACM or IEEE, to detect violations of these policies.
• By submitting your article to an IEEE Publication, you are hereby acknowledging that you and your co-authors are subject to all ACM Publications Policies, including ACM’s new Publications Policy on Research Involving Human Participants and Subjects . Alleged violations of this policy or any ACM Publications Policy will be investigated by ACM and may result in a full retraction of your paper, in addition to other potential penalties, as per ACM Publications Policy.
• Please ensure that you and your co-authors obtain an ORCID ID, so you can complete the publishing process for your accepted paper. ACM has been involved in ORCID from the start and we have recently made a commitment to collect ORCID IDs from all of our published authors. The collection process has started and will roll out as a requirement throughout 2023. We are committed to improve author discoverability, ensure proper attribution and contribute to ongoing community efforts around name normalization; your ORCID ID will help in these efforts.
• Authors’ names must be omitted from the submission.
• All references to the author’s prior work should be in the third person.
• While authors have the right to upload preprints on ArXiV or similar sites, they must avoid specifying that the manuscript was submitted to ICPC 2025.
• During review, authors should not publicly use the submission title. They should thus use a different paper title for any pre-print in arxiv or similar websites.
• Further advice, guidance, and explanation about the double-anonymous review process can be found in the Q&A page from ICSE .
• By submitting to the ICPC Research Track, authors acknowledge that they conform to the authorship policy of the ACM , and the authorship policy of the IEEE .

Submissions to the Technical Track that meet the above requirements can be made via the Research Track submission site by the submission deadline. Any submission that does not comply with these requirements may be desk rejected without further review.

Submission site: TBD

We encourage the authors to upload their paper info early (and can submit the PDF later) to properly enter conflicts for double-anonymous reviewing. It is the sole responsibility of the authors to ensure that the formatting guidelines, double anonymous guidelines, and any other submission guidelines are met at the time of paper submission.

Submissions from the ICPC 2025 General Chair, Program Chairs, their current students and current postdocs are not allowed to any ICPC 2025 track.

Open Science Policy

The research track of ICPC 2025 is striving to abide by Open Science policies. In summary, the steering principle is that all research results should be accessible to the public, if possible, and that empirical studies should be reproducible. In particular, we actively support the adoption of open data and open source principles and encourage all contributing authors to disclose (anonymized and curated) data to increase reproducibility and replicability. Note that sharing research data is not mandatory for submission or acceptance. However, sharing is expected to be the default, and non-sharing needs to be justified.

Upon submission to the research track, authors are asked: - to make their data available to the program committee (via upload of supplemental material or a link to an anonymous repository) – and provide instructions on how to access this data in the paper; or - to include in the paper an explanation as to why this is not possible or desirable; and - to indicate if they intend to make their data publicly available upon acceptance. - Supplementary material can be uploaded via the ICPC submission site or anonymously linked from the paper submission. Authors are asked to carefully review any supplementary material to ensure it conforms to the double-anonymous policy (described above). For example, code and data repositories may be exported to remove version control history, scrubbed of names in comments and metadata, and anonymously uploaded to a sharing site to support review. One resource that may be helpful in accomplishing this task is this blog post .

Review and Evaluation Criteria

Research papers will be reviewed by at least three members of the Program Committee. The reviewers will remain anonymous and signing of reviews will not be permitted in the review.

Submissions will be evaluated based on the following criteria:

• Soundness: The extent to which the paper’s contributions are supported by rigorous application of appropriate research methods;
• Significance: The extent to which the paper’s contributions impact program comprehension, and if needed, under which assumptions;
• Novelty: The extent to which the contributions are sufficiently original with respect to the state of the art and clearly explained and contrasted against it;
• Verifiability: The extent to which the paper includes sufficient information to support independent verification or replication of the paper’s claimed contributions;
• Presentation: The extent to which the paper’s quality of writing meets the standards of ICPC, including clear descriptions and explanations, appropriate use of the English language, absence of major ambiguity, clearly readable figures and tables, and adherence to the formatting instructions provided above.

Reviewers will carefully consider all of these criteria during the review process, and authors should take great care in clearly addressing them all. The paper should clearly explain the claimed contributions, and how they are sound, significant, novel, and verifiable, as described above.

Author Response Period

ICPC 2025 will offer an author response period. In this period the authors will have the opportunity to inspect the reviews, and answer specific questions raised by the program committee. This period is scheduled after all reviews have been completed, and serves to inform the subsequent decision-making process. Authors will be able to see the full reviews, including the reviewer scores as part of the author response process.

Withdrawing a Paper

Authors can withdraw their paper at any moment until the final decision has been made, through the paper submission system. Resubmitting the paper to another venue before the final decision has been made without withdrawing from ICPC 2025 first is considered a violation of the concurrent submission policy, and will lead to automatic rejection from ICPC 2025 as well as any other venue adhering to this policy.

Publication and Presentation

Upon notification of acceptance, all authors of accepted papers will receive further instructions for preparing the camera-ready versions of their submissions. If a submission is accepted, at least one author of the paper is required to register for ICPC 2025, attend the conference and present the paper in person . All accepted papers will be published in the conference electronic proceedings. The presentation is expected to be delivered in person, unless this is impossible due to travel limitations (related to, e.g., health, visa, or COVID-19 prevention). Details about the presentations will follow the notifications. The official publication date is the date the proceedings are made available in the ACM or IEEE Digital Libraries. This date may be up to two weeks prior to the first day of ICPC 2025. The official publication date affects the deadline for any patent filings related to published work. Purchases of additional pages in the proceedings is not allowed.

Coen De Roover

Vrije universiteit brussel.

Gema Rodríguez-Pérez

University of british columbia (ubc).