Tracks and important dates

Format and Submission – General Instructions 

Papers must not have been previously published, or be in consideration for, any journal, book, or other conference. Papers will be evaluated by members of the program committee based on their originality, technical soundness and presentation quality. Submissions must conform to Springer’s LNCS format.

Similar to previous editions of SSBSE, the Research Paper Track will follow a strong double blind reviewing process in which the identity of authors will not be known to the program committee at any time during the process. The papers submitted must not reveal the authors’ identities in any way.

• Authors should leave out author names and affiliations from the body of their submission.
• Authors should ensure that any citation to related work by themselves is written in third person, that is, ” the prior work of XYZ” as opposed to ” our prior work” .
• Authors should avoid providing URLs to author-revealing sites (tools, data sets). The paper can mention the existence of such sites, but the visit of such sites should not be needed to conduct the review. You can submit, optionally, a zip file or a pdf file containing supplementary material (raw data, data sets, experiments, etc.), which should also be blinded. The program committee will not necessarily consider it in the paper review process.
• Authors should anonymize author-revealing company names, yet can provide general characteristics of the organizations involved needed to understand the context of the paper.

Authors having further questions on double blind reviewing are encouraged to contact the Program Chairs by email. Papers that do not comply to the double blind review process will be desk-rejected.

If a paper is accepted, at least one author is expected to attend the symposium and to present the paper. In case of a student paper, the first (student) author is expected to attend and present the paper.

EMSE Special Issue

The authors of the best manuscripts will be invited to extend their papers into full journal papers, for the EMSE Special Issue on Advances in Search-Based Software Engineering.

Conference proceedings

Accepted papers will be published in a volume of the Springer Lecture Notes in Computer Science series (LNCS).

---

Research Paper Track

We invite the submission of high quality papers describing novel and original work in all areas of SBSE including, but not limited to, applications of SBSE to novel problems, theoretical analyses of search algorithms for software engineering, rigorous empirical evaluations of SBSE techniques, and reports of industrial experiences.

SSBSE welcomes  applications of search-based techniques throughout the software engineering lifecycle. We also invite the application of a broad range of search methods – ranging from exact operational research techniques to nature-inspired algorithms, local search metaheuristics, and simulated annealing.

We particularly encourage papers that report on software engineering applications to which SBSE has not been applied before.

Research papers must have up to 15 pages in length, including all text, figures, references and appendices.

Important dates

Abstract submission: June 1, 2020 (previously May 25, 2020)
Full Paper Submission: June 8, 2020 (previously June 1, 2020)
Initial decisions and notification (paper improvement process begins for selected papers): July 6, 2020 (previously June 19, 2020)
Revised paper submission for selected papers: July 13, 2020 (previously June 26, 2020)
Final decisions and notification: July 20, 2020 (previously July 3, 2020)
Camera-ready submission: July 27, 2020 (previously July 17, 2020)

 

---

Journal First Track

This year SSBSE welcomes authors of papers accepted in relevant journals to present and discuss their work directly with the community.

Scope

A journal-first presentation must adhere to the following criteria:

• The accepted journal paper was accepted to a journal from the list below no earlier than May 30, 2019 and no later than March 30, 2020
• The paper is in the scope of the conference
• The paper was published in a journal from the list below 
  • IEEE Transaction of Software Engineering (IEEE TSE),
  • Empirical Software Engineering (EMSE),
  • Journal of System and Software (JSS)
  • Information and Software Technology (IST)
  • ACM Transactions on Software Engineering and Methodology (ACM TOSEM),
  • Automated Software Engineering (ASE)

Submission

Authors of manuscripts that respect these criteria are invited to submit a one-page presentation proposal consisting of the paper title, authors, an extended abstract and a pointer to the original journal paper.

Evaluation

The presentation proposals are evaluated in terms of contributions to and relevance to the community. An award will be given to the outstanding paper.

Presentation

At least one author of each presentation accepted for the journal-first program must register and attend the conference to present the paper.

The journal-first manuscripts are published through the journals and will not be part of the SSBSE proceedings. The journal-first papers will be listed in the conference program with  and the proceeding will only contain the abstract and a pointer to the journal publication.

Important dates

Proposal submission: June 15, 2020 (previously June 1, 2020)
Author notification: July 13, 2020 (previously June 29, 2020)

 

---

NIER Track

The New Ideas and Emerging Results Track at SSBSE 2020 is a forum to present and get feedback on forward-looking, innovative ideas in the Search-Based Software Engineering (SBSE) field. This forum provides an opportunity for researchers to present and gather feedback on groundbreaking results, tools under development, experience reports, or position papers. The track provides an opportunity for researchers to introduce ideas that challenge the status-quo in the SBSE community, start a discussion, and receive feedback on new research ideas or results in an early stage of investigation. Early career researchers and Ph.D. students are particularly encouraged to present inspiring research efforts that push the boundaries of SBSE. However, researchers at all levels are invited to submit.

You are encouraged to submit your work that provides:

• Innovative results that may open new research directions in SBSE;
• New ideas that can be further explored by emerging SBSE community;
• Perspectives that call into question long held beliefs or conventions;
• Visions of new directions in SBSE; and,
• Novel interdisciplinary synergies.

Papers submitted to this track should address the following questions:

• Why is the problem worth exploring?
• What is the potential for disruption of current practice?
• What makes the approach original?
• Is the proposed approach sound and feasible?
• Has the relevant literature been covered?

The submissions should not exceed 6 pages in length and should be prepared for double-blind review following the symposium general instructions.

Important dates

Full Paper Submission: June 27, 2020 (previously June 8, 2020)
Author notification: July 13, 2020 (previously June 29, 2020)
Camera-ready submission: July 27, 2020 (previously July 17, 2020)

 

---

Challenge Track

Call for Solutions

We challenge everyone interested in SBSE to apply tools, techniques, and algorithms to our selected programs. We want you to apply your SBSE expertise to do interesting things to or with the software and uncover interesting things related to it.

The SSBSE Challenge Track, held in coordination with the SSBSE Symposium, is an exciting opportunity for SBSE researchers to apply tools, techniques, and algorithms on real-world software. Participants can use their SBSE expertise to carry out interesting analysis on open source development projects and uncover interesting insights related to them. The principal criterion is to produce interesting results

Submitting to the Challenge Track

In order to participate, you should:

• Focus your investigation on one of the selected programs (see below).
• Report your findings in a six-page paper using the regular symposium format. Note these results should have not been previously published in any peer-reviewed venue.
• Submit the challenge paper by the deadline.
• If your paper is accepted, present your findings at SSBSE 2020.

It is not mandatory for submissions to the SSBSE Challenge track to implement a new tool, technique, or algorithm. However, we do expect that applying your existing or new tools/techniques/algorithms to one of the challenge programs lead to relevant insights and interesting results.

Each submission must include a replication artifact aimed at helping reviewers make more informed decisions on acceptance and award. After the submission deadline, the track chairs will contact authors requesting a download link for their artifact package, which will then be shared with the assigned reviewers ensuring the review process remains double-blind.

The criteria for paper acceptance are the following:

• Application of an SBSE technique to one of the programs below.
• Technical soundness.
• Reproducibility (artifact evaluation).
• Readability and presentation.

Challenge Programs

Participants are invited to investigate and report upon one of the following open source projects. You are free to focus on any particular version or a comparison of different versions; you can also choose to analyse, test, improve, or apply any other SBSE-based activities to either parts or the whole of a project, including source code, documentation, or any other related resources (bug database, versioning histories, online discussions, etc.) that can be found freely available online.

LibreOffice https://www.libreoffice.org

LibreOffice is a large open-source productivity suite, implemented in several languages including C++, with a total of 8 MLOC. The project incorporates three levels of regression testing.

SQLite https://www.sqlite.org

SQLite is arguably the most popular database in the world. It is designed for efficiency, simplicity, and can be deployed as a single C source code file. The project incorporates 338 KLOC and three separately developed test suites.

Gson https://github.com/google/gson

Gson is a Java library that can be used to convert Java Objects into their JSON representation and to convert JSON strings to equivalent Java objects. Gson can work with arbitrary Java objects including pre-existing objects for which you do not have source-code.

Flask http://flask.pocoo.org

Flask is a very popular minimalist web framework for Python, with 9 KLOC. It comes with an extensive test suite.

MyExpenses https://github.com/mtotschnig/MyExpenses

MyExpenses is an Android app designed to keep track of your expenses and incomes, and to export them as QIF files into a desktop finance tool.

Submission Details

The papers must be at most six pages long in PDF format and should conform at time of submission to the SSBSE/Springer LNCS format and submission guidelines. They must not have been previously published, or be in consideration for, any journal, book, or other conference. Please submit your challenge paper to EasyChair on or before the Challenge track deadline. At least one author of each paper is expected to present it at SSBSE 2020. Papers for the Challenge track are also required to follow the double blind restrictions. All accepted contributions will be published in the conference electronic proceedings.

Challenge Track Jam Session

The SSBSE Challenge track is a good opportunity for new researchers to join the SBSE community and develop a taste and gain practical expertise in the field. It also allows researchers to apply techniques and tools to real-world software and to discover novel practical (or even theoretical) challenges for future work.

The CREST centre at UCL is a long-standing contributor of accepted papers to the Challenge Track. Their sustained success can be attributed in part to the organisation of a Jam Session in preparation for the Challenge Track submission deadline, as part of the CREST Open Workshops (COW) (See 2017 edition). This Jam Session runs over two consecutive days and is open to the public. Matheus Paixão and David White, former organisers of the session at CREST, kindly agreed to share their methodology with the goal of motivating other research groups to replicate their efforts in producing successful Challenge Track submissions:

1. The organiser of the session overviews the Challenge Track call (e.g., describing how challenge track papers differ from technical research papers, subject systems, prizes, format and deadline).
2. The organiser leads a technical discussion on the Challenge Track’s proposed systems, with emphasis on their amenability for SBSE techniques and tools.
3. Attendees brainstorm and propose ideas (potential Challenge Track submissions).
4. Ideas are discussed and refined collectively. Attendees sign up for the ones they find more interesting and feasible. A team is formed for each of the most promising ideas; the person who proposed the idea becomes the team leader.
5. Break out into teams to turn selected ideas into projects and work on them throughout the first day.
6. At the end of the first day, the audience reconvenes, each team reports on their progress, proposes a plan for the second day, and collects feedback.
7. Teams continue to work on their projects during the second day. Each team presents the status of their project at the end of the second day. Projects deemed infeasible are abandoned and team members may join other teams.
8. At the end of the two-day Jam Session, the team leader is in charge of leading the effort to ensure the project results in a submission to the SSBSE Challenge Track.

Further Information

If you have any questions about the challenge, please email the Challenge Track chairs.

Important dates

Abstract submission: June 15, 2020 (previously June 1, 2020)
Full Paper Submission: June 22, 2020 (previously June 8, 2020)
Author notification: July 13, 2020 (previously June 29, 2020)
Camera-ready submission: July 27, 2020 (previously July 17, 2020)

SUBMIT

 

Call for Challenge Cases

To collect relevant challenge cases for the next editions of SSBSE, this year we introduce a call for challenge cases. Submitted cases will be reviewed to ensure that the required information are clearly described. Accepted cases will be part of the official conference proceedings. Authors of the accepted cases must attend the challenge track and participate in the discussion. Next editions of SSBSE will call for solutions to the accepted cases, as well as for new challenge cases.

Submitting a proposal for Challenge Cases

A challenge case paper can present (1) a particular data set with specific questions or (2) a call for a solution to a particular problem in a given context. The following information is needed for both types of case descriptions:

For data sets with specific questions:

• What is your dataset and how was it obtained?
• What is its size?
• How can this data be accessed?
• Do you provide tools to process the data?
• Provide at least one concrete question you want participants to answer. You can provide multiple questions
• For each question, provide the criteria for evaluating an answer/solution

For call to solutions:

• What is the concrete problem you want participants to solve?
• How can a solution be evaluated? The following are some ideas on how to specify evaluation criteria:

• • Concrete evaluation metrics (e.g., precision, recall, accuracy etc. depending on the problem)
  • Concrete test cases participants can evaluate their solution against (e.g, provide input and expected output and participants are expected to provide a solution that gets from one point to the other)
  • A list of systems to evaluate their solution against (e.g., a list of C systems that have a large number of nested #ifdefs, because your problem only makes sense in the context of higher-order variability)
  • A reference implementation to compare against, according to particular metrics

Additional requirements for both types of cases:

• The description should contain the URL of a public repository or artifact page that contains all the instructions needed to get started with the case study
• Optional: Case authors may include a list of 5 names of researchers or practitioners who have the expertise required to evaluate submitted solutions. The case authors themselves may be part of this list. The challenge track co-chairs will consider this list when creating the SSBSE challenge program committee next year

Submission details

The papers must be at most six pages long in PDF format and should conform at time of submission to the SSBSE/Springer LNCS format and submission guidelines. Please submit your challenge case to EasyChair on or before the Challenge Case track deadline. At least one author of each paper is expected to present it at SSBSE 2020. All accepted contributions will be published in the conference electronic proceedings.

Further Information

If you have any questions about the challenge, please email the Challenge Track chairs.

Important dates (will be updated)

Abstract submission: June 29, 2020 (Previously June 22, 2020)
Full Paper Submission: July 6, 2020 (Previously June 29, 2020)
Author notification: July 20, 2020 (Previously July 6, 2020)
Camera-ready submission: July 27  (Previously July 17, 2020)