C语言辅导 | COMP9319 Assignment RLFM Index

C辅导编写程序要求能够高效地搜索RLFM编码的记录文件

COMP9319 2019T2 Assignment 2: RLFM Index (Run-Length Encoded BWT)

Your task in this assignment is to create a search program that implements BWT backward search, which can efficiently search a RLFM encoded record file. The original file (before RLFM) format is:

where <offset1>, <offset2>, <offset3>, etc. are integer values that are used as unique record identifiers;
and <text1>, <text2>, <text3>, etc. are record values (text), which include any ASCII alphabets with ASCII values from 32 to 126, tab (ASCII 9) and newline (ASCII 10 and 13). For simplicity, there will be no open or close square bracket in the record values.

Your C/C++ program, called rlebwt , accepts:

as commandline input arguments. The search term can be up to 512 characters. To make the assignment easier, we assume that the search is case sensitive.

If -a is specified, using the given query string, rlebwt will perform backward search on the given RLFM encoded file, and output the sorted and unique identifiers (no duplicates) of all the records that contain the input query string to the standard output. Each identifier is enclosed in a pair of square brackets, one line (ending with a ‘\n’) for each match.

If -m is specified, given a query string, rlebwt will output the total number of matching substrings (count duplicates) to the standard output. The output is the total number, with an ending newline character.

Similarly, rlebwt will output the total number of unique matching records (do not count duplicates) if -r is specified.

If -n is specified, using the given record identifier, rlebwt will output the original record value (text) to the standard output with a ‘\n’ at the end.

For any of the above options, if a match cannot be found, simply output nothing.

Although you do not need to submit a BWT encoder, it is a part of this assignment that you will implement a simple BWT encoding program based on RLFM (this will help you in understanding the lecture materials and assist in testing your assignment).

Sample files are provided in ~cs9319/a2/.

The file extensions represent their corresponding types:

For the B and B’ arrays, after the last bit is written to the file, fill in the gap (if any) of the last byte with bit 1. Check the xxd examples below for details.

Whenever rlebwt is executed using a given file FILENAME, for example:

where X can be any one of the options (-m, -r, -a, -n), it will take FILENAME.s and FILENAME.b as input; and also check if FILENAME.bb exists. If FILENAME.bb does not exist, it will generate one.

After that, it will check if INDEX_FOLDER exists. If not, it will create it as an index folder. Index files will then be generated inside this index folder accordingly.

In addition to the B’ array, your solution is allowed to write out up to 6 external index files that are in total no larger than the total size of the given, input FILENAME.s file plus 2 x the size of the given FILENAME.b. If your index files are larger than this limit, you will receive zero points for the tests that involve that given FILENAME. You may assume that the index folder (and its index files inside) will not be deleted during all the tests for a given FILENAME, and all the INDEX_FOLDER are uniquely and correspondingly named. Therefore, to save time, you only need to generate the index files when their folder does not exist yet.

Suppose the original file (say dummy.txt) before RLFM is:

Some examples:

In the above example, we assume dummy.s and dummy.b exist in the a2 folder of our home directory. rlebwt will generate dummy.bb inside a2, and will then create an index folder called dummyIndex (with the index files inside dummyIndex) inside a2 as well.

In the following example, we assume dummy.s and dummy.b exist in the XYZ folder of the account MyAccount. You will check if dummy.bb exists in ~MyAccount/XYZ/. If not, your submittedrlebwt will generate dummy.bb in ~MyAccount/XYZ/ (assume you have write permission in that folder). You will create an index folder called dummy (with the index files inside dummy) at your current directory.

Note that it is possible that your submission may be tested with the B’ files provided. For example, the RLFM encoded file path could be ~cs9319/a2/simple1 and path to index folder could be ~/a2/myIndex. Since simple1.bb is already there, you do not need to generate the B’ file again and just read and use it from ~cs9319/a2/. You will then generate the index folder called myIndex at your own a2 folder.

You may find the tool xxd useful to inspect the binary files correspond to the B and B’ arrays. For example, you may use xxd to inspect the provided sample files:

In particular, simple1.s has 11 characters. Therefore, there will be 11 ones in the array B that correspond to the 11 characters in simple1.s. Since all the zeros representing the duplicates of these 11 characters, you can observe that the last bit “1” is just a gap filler. This also means the original text file will contain 15 characters.

We will use the make command below to compile your solution. Please provide a makefile and ensure that the code you submit can be compiled on a CSE Linux machine, e.g., wagner. Solutions that have compilation errors will receive zero points for the entire assignment.

Your solution should not write out any external files other than the B’ file and the index folder with maximum six files inside. Any solution that writes out external files other than these files will receive zero points for the entire assignment.

Your solution will be marked based on space and runtime performance. Your soluton will not be tested against any RLFM encoded files with their original files that are larger than 160MB.

Runtime memory is assumed to be always less than 16MB. Runtime memory consumption will be measured by valgrind massif with the option --pages-as-heap=yes, i.e., all the memory used by your program will be measured. Any solution that violates this memory requirement will receive zero points for that query test. To help you started, your tutor will provide an overview on valgrind and makefile during the tutorial in week 5 or week 6.

Any solution that runs for more than 90 seconds on a machine with similar specification as wagner for the first query on a given RLFM file will be killed, and will receive zero points for the queries for that RLFM file. After that any solution that runs for more than 20 seconds for any one of the subsequent queries on that RLFM file will be killed, and will receive zero points for that query test. We will use the time command and count both the user and system time as runtime measurement.

Your source code will be inspected. Marks may be deducted if your code is very poor on readability and ease of understanding; your code does not follow the requirements of this specification document; or your code is written deliberately to avoid being accurately measured by valgrind.

This assignment is worth 100 points. Below is an indicative marking proportion for large vs small files. You may want to ensure your assignment working properly for small files before enhance it for larger files.

Bonus marks (up to 10 points) will be awarded for the solution that achieves 100 points and runs the fastest overall (i.e., the shortest total time to finish all the tests). Note: regardless of the bonus marks you receive in this assignment, the maximum final mark for the subject is capped at 100.

Deadline: Monday 5th August 12:00 (noon). Late submissions will have marks deducted from the maximum achievable mark at the rate of roughly 1% of the total mark per hour that they are late (i.e., 24% per day), and no submissions will be accepted after 3 days late.

Use the give command below to submit the assignment or submit via WebCMS3:

Please use “classrun” to check your submission to make sure that you have submitted all the necessary files.

The work you submit must be your own work. Submission of work partially or completely derived from any other person or jointly written with any other person is not permitted. The penalties for such an offence may include negative marks, automatic failure of the course and possibly other academic discipline. Assignment submissions will be examined both automatically and manually for such submissions.

Relevant scholarship authorities will be informed if students holding scholarships are involved in an incident of plagiarism or other misconduct.

Do not provide or show your assignment work to any other person – apart from the teaching staff of this subject. If you knowingly provide or show your assignment work to another person for any reason, and work derived from it is submitted you may be penalized, even if the work was submitted without your knowledge or consent. This may apply even if your work is submitted by a third party unknown to you.