How to add full text search to my static website?
2020-12-30
Categories: Programming
After building the website, I want to add search function. There are some ways to do it:
- ElasticSearch: setup and maintenance cost
- MongoDB: have to insert posts into database
- Google Custom Search Engine: ads?
So, is there any search engine written in Go?
Bleve
has more star and be maintained more often than riot
, I would like to give it a try first.
Backend
Looking at the documentation, there are three steps to add search to your website:
1func createOrOpenIndex(posts []*blog.Post, indexPath string) (bleve.Index, error) { 2 var index bleve.Index 3 if _, err := os.Stat(indexPath); os.IsNotExist(err) { 4 index, err = bleve.NewUsing(indexPath, bleve.NewIndexMapping(), scorch.Name, scorch.Name, nil) 5 if err != nil { 6 return nil, errors.Errorf("failed to create index at %s: %v", indexPath, err) 7 } 8 } else if err == nil { 9 index, err = bleve.OpenUsing(indexPath, nil) 10 if err != nil { 11 return nil, errors.Errorf("failed to open index at %s: %v", indexPath, err) 12 } 13 14 if err := deletePostsFromIndex(posts, index); err != nil { 15 return nil, err 16 } 17 } 18 return index, nil 19}
Index will be created when starting, so it must be updated if there is a post has been deleted:
1func deletePostsFromIndex(posts []*blog.Post, index bleve.Index) error { 2 count, err := index.DocCount() 3 if err != nil { 4 return errors.Errorf("failed to get number of documents in the index: %v", err) 5 } 6 7 query := bleve.NewMatchAllQuery() 8 searchRequest := bleve.NewSearchRequest(query) 9 searchRequest.Size = int(count) 10 searchResults, err := index.Search(searchRequest) 11 if err != nil { 12 return errors.Errorf("failed to find all documents in the index: %v", err) 13 } 14 for i := 0; i < len(searchResults.Hits); i++ { 15 uri := searchResults.Hits[i].ID 16 if !isContains(posts, uri) { 17 if err := index.Delete(uri); err != nil { 18 return err 19 } 20 } 21 } 22 return nil 23}
1func indexPost(post *blog.Post, batch *bleve.Batch) error { 2 doc := document.Document{ 3 ID: post.URI, 4 } 5 if err := bleve.NewIndexMapping().MapDocument(&doc, post); err != nil { 6 return errors.Errorf("failed to map document: %v", err) 7 } 8 9 var b bytes.Buffer 10 enc := gob.NewEncoder(&b) 11 if err := enc.Encode(post); err != nil { 12 return errors.Errorf("failed to encode post: %v", err) 13 } 14 15 field := document.NewTextFieldWithIndexingOptions("_source", nil, b.Bytes(), document.StoreField) 16 if err := batch.IndexAdvanced(doc.AddField(field)); err != nil { 17 return errors.Errorf("failed to add index to the batch: %v", err) 18 } 19 return nil 20}
1func (ps *postService) Search(value string) ([]*blog.Post, error) { 2 query := bleve.NewMatchQuery(value) 3 request := bleve.NewSearchRequest(query) 4 request.Fields = []string{"_source"} 5 searchResults, err := ps.index.Search(request) 6 if err != nil { 7 return nil, errors.Errorf("failed to execute a search request: %v", err) 8 } 9 10 var searchPosts []*blog.Post 11 for _, result := range searchResults.Hits { 12 var post *blog.Post 13 b := bytes.NewBuffer([]byte(fmt.Sprintf("%v", result.Fields["_source"]))) 14 dec := gob.NewDecoder(b) 15 if err = dec.Decode(&post); err != nil { 16 return nil, errors.Errorf("failed to decode post: %v", err) 17 } 18 searchPosts = append(searchPosts, post) 19 } 20 21 return searchPosts, nil 22}
bleve CLI can be used to query the index:
1$ bleve query posts.bleve "linux" 25 matches, showing 1 through 5, took 625.866µs 3 1. 2019/09/19/about (0.318970) 4 Content 5 …developer. Then my passionate on Linux and open source led me to a different direction, a system administrator. 6With a strong background in Linux system administration, I was gradually moving to backe… 7 2. 2019/10/30/docker-exited-127 (0.233462) 8 Content 9 …/span> 10<span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#495050">5</span>COPY build/linux/<binary> . 11<span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#495050">6</span> 12…
Frontend
Add a search box using forms:
1<form class="form-inline my-2 my-lg-0" action="/search"> 2 <input class="form-control mr-sm-2" type="search" placeholder="Search" aria-label="Search" name="q"> 3 <button class="btn btn-outline-success my-2 my-sm-0" type="submit">Search</button> 4</form>
The action attribute defines where the data gets sent:
1 s.router.HandleFunc("/search", s.Error(s.searchHandler))
searchHandler will:
- parse form
- get the form value
- perform search
- render posts
References:
Tags: bleve full-text search golang
Related Posts:
- How to create snippets in Helix?
- A terminal UI for Taskwarrior
- A simple terminal UI for ChatGPT
- Learning how to code
- gocloud - writing data to a bucket: 403