\u003C/a>\u003C/figure>\u003C/div>\n\n\u003C!-- /wp:image -->\n\n\u003C!-- wp:paragraph -->\n\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\nLuckily, the team did not have to start from scratch. Prof. Roy brought Rodrigo to the University of Saskatchewan as a visiting research student. One of Prof. Roy’s students at the University of Saskatchewan was Masud Rahman, a PhD candidate in computer science, who had previously researched translating a programming task written in natural language into a list of relevant API classes with the help of Stack Overflow. The team was able to build on two previous tools, RACK and NL2API, developed under Prof. Roy’s lead with Masud as the core researcher. \u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\nIn order to reduce the gap between the queries and solutions, the team trained a word-embedding model with \u003Ca href=\"https://fasttext.cc/\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\">FastText\u003C/a>, using millions of Q&A threads from Stack Overflow as the training corpus. CROKAGE also expanded the natural language query (task description) to include unique open source software library and function terms, carefully mined from Stack Overflow.\u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\nThe team of researchers combined four weighted factors to rank the candidate answers. They made use of the traditional information retrieval (IR) metrics such as \u003Ca href=\"https://en.wikipedia.org/wiki/Tf%E2%80%93idf\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\">TF-IDF\u003C/a> and asymmetric relevance. To tailor the factors to Stack Overflow, they also adopted specialized ranking mechanisms well suited to software-specific documents. \u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\nIn particular, they collected the programming functions that potentially implement the target programming task (the query), and then promoted the candidate answers containing such functions. They hypothesized that an answer containing a code snippet that uses the relevant functions and is complemented with a succinct explanation is a strong candidate for a solution. \u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:image {\"id\":12201} -->\n\u003Cfigure class=\"wp-block-image\">\u003Cimg src=\"https://stackoverflow.blog/wp-content/uploads/2019/08/approach.png\" alt=\"\" class=\"wp-image-12201\"/>\n\n\u003Cfigcaption>Schematic diagram of CROKAGE. A) Corpus Preparation, B) Building \u003Cbr>Models, Maps and Indices, C) Searching for Relevant Answers, and D) Composition of Programming Solutions\u003C/figcaption>\n\n\u003C/figure>\n\u003C!-- /wp:image -->\n\n\u003C!-- wp:paragraph -->\nTo ensure that the written explanation was succinct and valuable, the team made use of natural language processing on the answers, ranking them most relevant by the four weighted factors. They selected programming solutions containing both code snippets and code explanations, unlike \u003Ca href=\"https://dl.acm.org/citation.cfm?id=3155650\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\">earlier\u003C/a>\u003Ca href=\"https://ieeexplore.ieee.org/document/6747170\"> studies\u003C/a>. The team also discarded trivial sentences from the explanations. CROKAGE uses the following two patterns to identify the important sentences:\u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:image {\"align\":\"center\",\"width\":368,\"height\":139} -->\n\n\u003Cdiv class=\"wp-block-image\">\u003Cfigure class=\"aligncenter is-resized\">\u003Cimg src=\"https://lh5.googleusercontent.com/eHNqJ295imoR0Whe9CpE-SHoWu31Jp6sxSPUpDl38urwYtt7l_2vLGp5F92sSznp6fb9YA3mlhWO6uhjUdXj4X92d0pieMjG1M8gz9LnkIFze2WkNMO2Sw9feTcMNQgb_IbI7ww\" alt=\"\" width=\"368\" height=\"139\"/>\u003Cfigcaption>\u003Cbr>\u003C/figcaption>\u003C/figure>\u003C/div>\n\n\u003C!-- /wp:image -->\n\n\u003C!-- wp:paragraph -->\nThese patterns ensure that each sentence has a verb, which is associated with a subject or an object. The first pattern guarantees that a verb phrase is followed by a noun phrase, while the second pattern guarantees that a noun phrase is followed by a verb phrase. They also ensure that a verb phrase is not a personal pronoun. According to \u003Ca href=\"https://dl.acm.org/citation.cfm?id=3107656.3107727\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\">the literature\u003C/a>, sentences with the above structures typically provide the most meaningful information. In order to synthesize a high-quality code explanation, they discard several frequent but trivial sentences (e.g.,“Try this:”, “You could do it like this:”, “It will work for sure”, “It seems the easiest to me” or “Yes, like doing this”). \u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\nLet us consider a search query -- “Convert between a file path and url”. CROKAGE delivers the solutions shown in the figure below. Note that the tool provides a ranked list of questions and answers. Each of these solutions contains not only a relevant code snippet but also its succinct explanation. Such a combination relevant code and corresponding explanation is very likely to help a developer understand both the solution to their problem and how best to implement that code in practice. \u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:image {\"align\":\"center\"} -->\n\n\u003Cdiv class=\"wp-block-image\">\u003Cfigure class=\"aligncenter\">\u003Cimg src=\"https://lh4.googleusercontent.com/N5qYVT9axgVqCGwM7eYg1cegQE6PZGZlsFaFBX4Q98WySb3JGTIcKDNXyEczttM0LkGxAI4g5fLGJnnzhgsLowoPt69omI1xd6OD4YYWZXCskjYf33o91V3y5dLFvp4PauUxr1LlIfssFhqW\" alt=\"\"/>\u003Cfigcaption>\u003Cbr>\u003C/figcaption>\u003C/figure>\u003C/div>\n\n\u003C!-- /wp:image -->\n\n\u003C!-- wp:paragraph -->\nThe team analyzed the results of 48 programming queries processed by CROKAGE. The results outperformed six baselines, including the state-of-art research tool, \u003Ca href=\"https://github.com/tkdsheep/BIKER-ASE2018\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\">BIKER\u003C/a>. Furthermore, the team surveyed 29 developers across 24 coding queries. Their responses confirm that CROKAGE produces better results than that of the state-of-art tool in terms of relevance of the suggested code examples, benefit of the code explanations, and the overall solution quality (code + explanation). \u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:image {\"id\":12202} -->\n\u003Cfigure class=\"wp-block-image\">\u003Cimg src=\"https://stackoverflow.blog/wp-content/uploads/2019/08/comparison1.png\" alt=\"\" class=\"wp-image-12202\"/>\n\n\u003Cfigcaption>Programming solutions for (a) AnswerBot (b) BIKER (c) CROKAGE\u003C/figcaption>\n\n\u003C/figure>\n\u003C!-- /wp:image -->\n\n\u003C!-- wp:paragraph -->\nTo be sure, CROKAGE still has some limitations. For instance, if the query is poorly formulated, the tools will not suggest on how to improve the query. Like any other search tool, the results, though encouraging, are not perfect. The team is still investigating other factors that could not only help find higher quality answers, but also improve the synthesized solution offered up as a final result. \u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\nIf you would like to try CROKAGE for yourself, the service is experimentally available at \u003Ca rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\" href=\"http://www.isel.ufu.br:9000\" target=\"_blank\">http://www.isel.ufu.br:9000\u003C/a>. It’s limited to Java queries for now, but the creators hope to have an expanded version open to the public soon. \u003Cbr>\u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\nMore details about the tool can be found in the original \u003Ca rel=\"noreferrer noopener\" href=\"https://dl.acm.org/citation.cfm?id=3339130\" target=\"_blank\">paper\u003C/a>. If you are interested in project source code, the authors provided a replication package available at \u003Ca rel=\"noreferrer noopener\" href=\"https://github.com/muldon/CROKAGE-replication-package\" target=\"_blank\">GitHub\u003C/a>. \u003Cbr>\u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\nThe team would also like to acknowledge the work that went into tools that proceeded CROKAGE - \u003Ca rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\" href=\"https://dl.acm.org/citation.cfm?id=3238191\" target=\"_blank\">BIKER\u003C/a>, \u003Ca rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\" href=\"https://link.springer.com/article/10.1007%2Fs10664-018-9671-0\" target=\"_blank\">RACK\u003C/a>, and \u003Ca href=\"https://ieeexplore.ieee.org/document/8530053\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\">NLP2API\u003C/a> - without which their current work would not exist. \u003Cbr>\u003Cbr>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\nDisclaimer: As CROKAGE is a research project deployed on a university lab server, it may suffer from some network instability and server overload.\u003Cbr>\n\u003C!-- /wp:paragraph -->","html","2019-08-14T14:00:20.000Z",{"current":292},"crokage-a-new-way-to-search-stack-overflow",[294,301,306,311,316,321,326,331,335,339,344],{"_createdAt":295,"_id":296,"_rev":297,"_type":298,"_updatedAt":295,"slug":299,"title":200},"2023-05-23T16:43:21Z","wp-tagcat-biker","9HpbCsT2tq0xwozQfkc4ih","blogTag",{"current":300},"biker",{"_createdAt":295,"_id":302,"_rev":297,"_type":298,"_updatedAt":295,"slug":303,"title":305},"wp-tagcat-bulletin",{"current":304},"bulletin","Bulletin",{"_createdAt":295,"_id":307,"_rev":297,"_type":298,"_updatedAt":295,"slug":308,"title":310},"wp-tagcat-code-for-a-living",{"current":309},"code-for-a-living","Code for a Living",{"_createdAt":295,"_id":312,"_rev":297,"_type":298,"_updatedAt":295,"slug":313,"title":315},"wp-tagcat-crokage",{"current":314},"crokage","CROKAGE",{"_createdAt":295,"_id":317,"_rev":297,"_type":298,"_updatedAt":295,"slug":318,"title":320},"wp-tagcat-crowd-knowledge-answer-generator",{"current":319},"crowd-knowledge-answer-generator","Crowd Knowledge Answer Generator",{"_createdAt":295,"_id":322,"_rev":297,"_type":298,"_updatedAt":295,"slug":323,"title":325},"wp-tagcat-machine-learning",{"current":324},"machine-learning","machine learning",{"_createdAt":295,"_id":327,"_rev":297,"_type":298,"_updatedAt":295,"slug":328,"title":330},"wp-tagcat-nl2api",{"current":329},"nl2api","NL2API",{"_createdAt":295,"_id":332,"_rev":297,"_type":298,"_updatedAt":295,"slug":333,"title":257},"wp-tagcat-rack",{"current":334},"rack",{"_createdAt":295,"_id":336,"_rev":297,"_type":298,"_updatedAt":295,"slug":337,"title":338},"wp-tagcat-search",{"current":338},"search",{"_createdAt":295,"_id":340,"_rev":297,"_type":298,"_updatedAt":295,"slug":341,"title":343},"wp-tagcat-stackoverflow",{"current":342},"stackoverflow","Stackoverflow",{"_createdAt":295,"_id":340,"_rev":297,"_type":298,"_updatedAt":295,"slug":345,"title":343},{"current":342},"CROKAGE: A New Way to Search Stack Overflow",[348,354,359,364],{"_id":349,"publishedAt":350,"slug":351,"sponsored":12,"title":353},"1d082483-6dc6-424b-8b09-9c84b54779da","2025-09-02T17:00:00.000Z",{"_type":10,"current":352},"back-to-school-developers-at-stack-overflow-have-some-advice-for-you","Back to school? Developers at Stack Overflow have some advice for you",{"_id":355,"publishedAt":350,"slug":356,"sponsored":12,"title":358},"5cd91820-9515-4be5-87ae-e919fd443c18",{"_type":10,"current":357},"getting-started-on-stack-overflow-a-step-by-step-guide-for-students","Getting started on Stack Overflow: a step-by-step guide for students",{"_id":360,"publishedAt":350,"slug":361,"sponsored":12,"title":363},"614538a9-c352-4024-adf1-fa44a9f911b6",{"_type":10,"current":362},"stack-overflow-is-helping-you-learn-to-code-with-new-resources","Stack Overflow is helping you learn to code with new resources",{"_id":365,"publishedAt":350,"slug":366,"sponsored":12,"title":368},"763b1d36-83d8-4178-9c2d-32d705ea1d7b",{"_type":10,"current":367},"introducing-your-newest-study-buddy-stackoverflow-ai","Introducing your newest study buddy: stackoverflow.ai",{"count":370,"lastTimestamp":371},22,"2023-05-25T09:46:48Z",["Reactive",373],{"$sarticleModal":374},false,["Set"],["ShallowReactive",377],{"sanity-LiNQ9yTZsluW0xvUaYTnFmeljQmvvzclMhNfsiJeBqo":-1,"sanity-comment-wp-post-12194-1756998712494":-1},"/2019/08/14/crokage-a-new-way-to-search-stack-overflow"]